Hi,
Looks like a node.js driver was available at some point,
Yet I can’t find it neither in the distribution nor online.
Any pointers?
We did have Node.js bindings, but they had not been maintained to support versions later than 0.10. As a result, we decided to remove them from the repo, though they still exist in history for anyone who wants to work on reviving them.
There has been some interest shown in doing that, and you can see the discussion here:
Hi!
I’ve spent the last couple of days starting to revive the old bindings. The discussion about it is here and the code is here.
I have the library building and working, but I want to make a lot of changes to modernise the library. For example, make transactions use promises so you can avoid callback hell and just:
db.doTransaction(async tn => {
const a = await tn.get('foo')
const b = await tn.get('bar')
tn.set('foobar', a + b)
})
and have that work.
There’s a few design issues - like I want to remove the version agnostic aspect of the library and lean on npm to do that for us, but thats controversial. Anyway, I should have something usable in the next few days - a week.
Just an update: I’ve got basic KV support working. I decided to take the opportunity to rip the old node module to pieces and rebuild it, because there’s so much old stuff in there. (Like a hand-rolled futures implementation. Hehehe).
The module lives here:
It works, but again, only for the basic KV operations. I’ll add layers over the next few weeks. There’s probably also some new bugs - please give it a try and file issues if you find them. Bear in mind the API will churn over the next few weeks. Don’t bet the farm on this library yet - it needs a lot more testing before I’m happy with it.
The most annoying thing about it at the moment is that you need to have a copy of the foundationdb native libraries on any machine you deploy to. I’d like to fix this - but I want to get all the basic functionality in first.
Update:
Features
- I’ve added range support and key selectors so standard key value transactions are feature complete now.
- I’ve mostly finished reviving triple support (missing version stamps still but everything else is working).
- Option passing works, with options generated from the XML file in fdb source. Not documented yet though.
- The only main functionality that’s missing now is the Subspace and Directory layers.
From a correctness standpoint:
- The library builds and works on windows, Mac and Linux, under node 4 through 10 and electron. No CI yet though.
- I have a basic unit test suite for the API
- Valgrind and ASAN both run and seem reasonably happy. I’d like to do a more thorough leak test once we have more testing
- I’m just starting to port the fuzzer over now. This is a really impressive piece of testing infrastructure. I’m super jazzed by how it’s designed.
I would like help with:
- Documentation. Bringing the node docs up to the quality standard of the other official bindings will be a lot of work. We should be able to use the python and ruby docs as a starting point and port the specifics to node. I’m not being paid for this work, and unless that changes I’m not the person who will do this super valuable doc work.
- API feedback. There’s a few important design decisions I’m making with the library, like how I’m abandoning the client specified API version in favour of binding the api version to the node module version. And the type (number or buffer) of versions? And how should the library respond by default if you pass a number as a key or value? Should there be a standard way to fetch strings? I’d love some feedback before it’s all baked in stone!
1 Like
After checking out the other bindings, I think I’m going to go ahead and implement something using the Node bindings because I’m most comfortable in the JS ecosystem. I’ll pop in and raise an issue if I start working on something.
I will probably start with Versionstamp keys and then move on to the Tuple layer unless you get to that first.
1 Like
If you pin the API version, you should take on responsibility for doing your best to protect developers from unexpected changes to the API. It’s not that npm isn’t a perfectly legitimate way to pin a version, it’s just that you need to think hard about what people will actually do. For example, you probably need to increase the major version number every time the API version changes, so that package.json dependency version declarations starting with “^” which I guess are the default for --save will not pick up API version changes automatically.
The way I see it, the API that the user consumes is a combination of FDB API version + binding version. The idea that users specify two numbers is weird. But yeah absolutely, the version number should be incremented through standard semver versioning on the package whenever compatibility on either package is broken.
Do fdb versions conform to semver? That is, should all code written against 510 work unmodified with API version 520? Or should I just bump the major version number with each FDB API version bump?
" the API that the user consumes is a combination of FDB API version + binding version"
Yes, I think this is a perfectly valid model.
You should bump the major version number whenever you change the API version you pass. But you don’t necessarily have to change API versions for every minor FDB release - often there is no difference. The release notes should make it clear.
FDB versions don’t really conform to semver, because the API versioning thing means that FDB, per se, basically never breaks API compatibility. So if we were semver I guess it would be 1.51.0 or something. Conversely, the entire API version should be viewed as a semver major version, which looks sort of like the FDB version only for convenience. It’s theoretically possible that patching a bug could involve a backward incompatible API change and a new API version 511. It appears that this actually happened with API version 101. (I can’t believe I looked that up.)
I had a long think about it the last couple of days and I’ve changed my mind on the version thing. I’ve just pushed version 0.4.0 of the library which re-adds a mandatory setAPIVersion() method just like the python, ruby, go and java bindings. I wrote up the motivation in the readme.
0.4.0 Changes
- The node library now passes lots of iterations of the API bindingtester. (It might pass more - I haven’t run it that long yet).
- I now consider it stable enough to use in production, though the API will probably continue to evolve a little over the next few weeks. There is also a chance some memory leaks remain. Please test!
The only big ticket features before 1.0 are:
- Documentation. I have some basic usage docs in the readme, but we should have a proper document which defines the API semantics for every function.
- The directory layer. We have the old code, but it needs to be ported to typescript (and massively simplified).
- Add binary prebuilds so you don’t need a local development environment to use the library. This is especially an issue on windows or for deployment.
Have you thought on how to fix the redistribution part of the fdb_c binaries alongside the binding? I’m faced with similar issues with the .NET binding, and I’m still unsure on what is the best solution.
One possibility would be to have the “core” binding available has a package versioned to follow the FoundationDB new APIs, and then have another “binary” package that redistributes some version of the fdb_c client, plus a dependency to the core binding package. But I’m not sure on what minimum/maximum version range should tie both packages together:
- You can use an older version of the core binding with a more recent
fdb_cversion (tied to the version of your cluster and for db bugfix) - You can also use a newer version of the core biding (bugfixes, some changes related to the underlying node.js or .NET CLR version, etc…) and rely on
setAPIVersion(..)to protect your app against unwanted behavior changes.
One solution is of course to have both packages independent from each other, but this does seem a bit of a bummer to not have a single “end point” to have a fully working solution for local development that just works out of the box.
1 Like
I wrote a reply, then realised we probably want to involve others in the conversation.
See here - How do bindings get an appropriate copy of `fdb_c`?