Is anyone able to mock transactions when using the golang bindings? How are people testing code when foundationdb is involved? Since it doesn’t use database/sql, there’s no easy path to injecting a mock driver and using that.
I want to overwrite things like fdb.Database.Transact and fdb.Transaction.Get but those aren’t set up in a way that they can be overridden (i.e. properties of the struct vs functions).
What approaches are people taking for mocks?
I test against an in memory single process cluster on the same machine with the node bindings. I don’t personally think mocking FDB out is a great idea, as you’d have to 100% match the semantics and not just the interface to be correct.
Thanks for the insight. I thought about spinning up fdb in CI as an option but in this case, it’s not my desired path as spinning up fdb from scratch (even via docker) is more time consuming than I want.
Since my use-case for FDB is as a key/value store, the interfaces I’d have to mock are pretty limited. The bindings in golang don’t seem to be designed with testing in mind (see lack of tests in the source) so it’s been a bit of a challenge. Implementing wrapper functions for the interfaces I use has been my hacky way so far. For me, it’s better than spinning up the db during each test suite run.
The bindings are tested using the bindingtester, which is a standardized test for all bindings which covers the entirety of the functionality by generating a bunch of operations and testing that they match what is expected.
I totally understand what you’re saying since go is a bit picky compared to how it works in Node, since I could just make mocks however I like.
Ah, thanks for the clarification. It’s definitely a golang peculiarity as other languages are more flexible when it comes to overriding methods. I’m going to work around it for my use case with some wrapper functions.
A trick that may help in some circumstances is to create a transaction, set the read version to 1, and then clear the whole database. You can then use that transaction sort of like a database, and then discard it at the end of a test. This will work without any real database to connect to, because your reads can always be supplied by the read your writes cache.
Thanks @dave! I may be able to give that a shot. Right now, I’ve been using a wrapper with a map[string]interface{} to hold keys and values and it’s working pretty well. Like go-sqlmock, I set up some expectations for certain keys to return certain values and then when I execute the wrapper function in mock mode, it pulls back the values from the map. I haven’t implemented anything other than tr.Get(), tr.Set(), and tr.Add() at the moment.
It’s definitely not as clean as implementing a driver for an existing database wrapper since I have to replace all my calls with wrapper functions so I implemented helpers for things like “does key exist”, “increment key”, “set key”, etc with transaction timeouts and retries and some other boilerplate that tended to make using the golang bindings duplicative either way.
Appreciate the suggestions. I’ll keep plugging a way on a couple of approaches and see what works best for a balance of CI speed and correctness of tests.
I have developed a thin layer on top of FoundationDB to replace it with BBolt in my testing environment. As @ryanworl said, it is not the ideal path, but so far I haven’t experienced unexpected behaviours between both environment. It is also probably worth noting that I don’t use all features provided by FoundationDB. (e.g. Watch)
It is on the github.com/stairlin/storage.go/kv package.
@basgys nice! Having a K/V package that can accept different drivers and is well maintained would be huge. database/sql does an awesome job of this in the SQL world and it’d be incredibly useful to have something similar in the K/V world. Hopefully that project, or similar will take off (I do worry about it’s goal to support graph, k/v, etc from the start…I’ve always had more success starting small and focused).
Thanks, @dkoston! I understand your concern about thinking too big. However, I usually work on another whenever Stairlin (organisation) needs it. Regarding the GraphDB layer, it was fairly trivial to implement as it is just a wrapper around Cayley.
Now back to the K/V store, the goal was not about having a generic K/V package with as many drivers as possible. It would not be a bad idea though. The only problem I see is that this K/V package is assuming an ordered key/value store with transaction support, which limits already the choices. Also, the API is virtually identical to FoundationDB’s, which is perhaps not the best choice for a generic K/V API.
If you are interested in using this package, I am open to including more drivers or other ideas in the project. Then you can benefit from the new extra layers added.