@KrysFR (.NET binding author) and I (NodeJS binding author) are both trying to figure out how clients should get an appropriate copy of fdb_c. I think this is more widely relevant, so I’m bumping the conversation to a new topic.
Ideally, you should just be able to npm install --save foundationdb in a node app (or equivalent in C#) and everything should just work. There shouldn’t be any other system-wide configuration to do, development libraries to install or anything else to worry about.
The question then is, how does the client get a copy of fdb_c which knows how to talk to the version of foundationdb you’re running in production? Apparently despite emulated support for multiple API versions, the network protocol itself frequently changes. fdb_c only knows how to talk to a specific version of the foundationdb cluster itself. When upgrading foundationdb in production you need to make multiple copies of the fdb_c library available to the client, so it can try all of them.
One possibility would be to have the “core” binding available has a package versioned to follow the FoundationDB new APIs, and then have another “binary” package that redistributes some version of the fdb_c client, plus a dependency to the core binding package. But I’m not sure on what minimum/maximum version range should tie both packages together:
- You can use an older version of the core binding with a more recent fdb_c version (tied to the version of your cluster and for db bugfix)
- You can also use a newer version of the core biding (bugfixes, some changes related to the underlying node.js or .NET CLR version, etc…) and rely on
setAPIVersion(..)to protect your app against unwanted behavior changes.One solution is of course to have both packages independent from each other, but this does seem a bit of a bummer to not have a single “end point” to have a fully working solution for local development that just works out of the box.
Yeah I’m not sure the best way to do this either. I thought about doing something like that - have two packages in the node package manager:
- A bundled copy of
fdb_c - The javascript binding code, which depends on the fdb_c package.
The problem with that is that when you upgrade your foundationdb database cluster itself, you want your client application to have access to multiple copies of fdb_c - one for the old version and one for the new version of foundationdb you’re deploying. And that means even if I bundled fdb_c into a node package you’d actually want it to depend on multiple copies of that package in order to migrate your database - which is something npm doesn’t support. Npm also doesn’t support platform/architecture-specific downloads, so the package would have to either bundle copies of fdb_c for every supported platform, or download the library itself at runtime. 
.
A different approach would be to just have a minimal fdb_c-specific ‘package repository’ sitting somewhere that all the bindings use. At launch the client application connects to foundationdb, queries for the cluster version, then downloads & caches an appropriate copy of fdb_c to connect. It uses a known, hardcoded https server address, ideally with pinned ssl keys & SHAs. The client could explicitly specify other versions (through environment variables or options) to preload fdb_c before the database is upgraded or for zero-downtime deployments.
For nodejs we could do the downloading in a postinstall hook, so it happens when you run npm install. If we did that users would need to add "fdb_preload_versions": [515, 517, 520] to their package.json. Then we slurp down appropriate copies of fdb_c when all the other dependancies are downloaded. This would work great for dockerised builds.
Another idea would be to compile fdb_c to webassembly / into a .NET managed DLL. Then we could distribute portable versions that worked across all operating systems, though it would still be deeply tied to the cluster version. (And in this case, language.) But it would make bundling it much easier.
Its a shame the network protocol is so incompatible, though I sort of see the justification. Does anyone have any other thoughts or ideas? I think I have a weak preference for the public fdb_c download repository at the moment, as messy as it would be.
