FoundationDB

How do bindings get an appropriate copy of `fdb_c`?

bindings

(Seph Gentle) #1

@KrysFR (.NET binding author) and I (NodeJS binding author) are both trying to figure out how clients should get an appropriate copy of fdb_c. I think this is more widely relevant, so I’m bumping the conversation to a new topic.

Ideally, you should just be able to npm install --save foundationdb in a node app (or equivalent in C#) and everything should just work. There shouldn’t be any other system-wide configuration to do, development libraries to install or anything else to worry about.

The question then is, how does the client get a copy of fdb_c which knows how to talk to the version of foundationdb you’re running in production? Apparently despite emulated support for multiple API versions, the network protocol itself frequently changes. fdb_c only knows how to talk to a specific version of the foundationdb cluster itself. When upgrading foundationdb in production you need to make multiple copies of the fdb_c library available to the client, so it can try all of them.

@KrysFR said:

One possibility would be to have the “core” binding available has a package versioned to follow the FoundationDB new APIs, and then have another “binary” package that redistributes some version of the fdb_c client, plus a dependency to the core binding package. But I’m not sure on what minimum/maximum version range should tie both packages together:

  • You can use an older version of the core binding with a more recent fdb_c version (tied to the version of your cluster and for db bugfix)
  • You can also use a newer version of the core biding (bugfixes, some changes related to the underlying node.js or .NET CLR version, etc…) and rely on setAPIVersion(..) to protect your app against unwanted behavior changes.

One solution is of course to have both packages independent from each other, but this does seem a bit of a bummer to not have a single “end point” to have a fully working solution for local development that just works out of the box.

Yeah I’m not sure the best way to do this either. I thought about doing something like that - have two packages in the node package manager:

  1. A bundled copy of fdb_c
  2. The javascript binding code, which depends on the fdb_c package.

The problem with that is that when you upgrade your foundationdb database cluster itself, you want your client application to have access to multiple copies of fdb_c - one for the old version and one for the new version of foundationdb you’re deploying. And that means even if I bundled fdb_c into a node package you’d actually want it to depend on multiple copies of that package in order to migrate your database - which is something npm doesn’t support. Npm also doesn’t support platform/architecture-specific downloads, so the package would have to either bundle copies of fdb_c for every supported platform, or download the library itself at runtime. :face_vomiting:.


A different approach would be to just have a minimal fdb_c-specific ‘package repository’ sitting somewhere that all the bindings use. At launch the client application connects to foundationdb, queries for the cluster version, then downloads & caches an appropriate copy of fdb_c to connect. It uses a known, hardcoded https server address, ideally with pinned ssl keys & SHAs. The client could explicitly specify other versions (through environment variables or options) to preload fdb_c before the database is upgraded or for zero-downtime deployments.

For nodejs we could do the downloading in a postinstall hook, so it happens when you run npm install. If we did that users would need to add "fdb_preload_versions": [515, 517, 520] to their package.json. Then we slurp down appropriate copies of fdb_c when all the other dependancies are downloaded. This would work great for dockerised builds.


Another idea would be to compile fdb_c to webassembly / into a .NET managed DLL. Then we could distribute portable versions that worked across all operating systems, though it would still be deeply tied to the cluster version. (And in this case, language.) But it would make bundling it much easier.

Its a shame the network protocol is so incompatible, though I sort of see the justification. Does anyone have any other thoughts or ideas? I think I have a weak preference for the public fdb_c download repository at the moment, as messy as it would be.


Node.js support
(Seph Gentle) #2

Ping! I’d love to hear some more people’'s thoughts on this

Is there a way to ask the foundationdb server what version of the client fdb_c file to connect with? Where is that code?


(A.J. Beamon) #3

If you have access to the server binary, then you can run fdbserver -v to get the version. The fdb_c version needs to match (at least in major and minor version).

If you don’t have access to the server binary but want to find out the version of a running cluster, you basically need to connect to it with a compatible client. If a client connects, then the cluster is running roughly the same version. You can get the exact version of each server process by getting the json status (e.g. by running status json in fdbcli, or by reading the key \xff\xff/status/json). If you wanted to try multiple client versions at once, you could use the multi-version client to achieve that.


(Seph Gentle) #4

Ok thanks. So, imagine a client which is written against version 500 of the API. I want a way for that client to be able to run unmodified throughout cluster upgrades, and without needing to manually provision new library files to the instance. If all clients need to be manually reprovisioned with new binaries every time the cluster is upgraded it removes most of the benefit of having a multiversion API.

So I propose we lock down the initial version header in the network API. I want a client to be able to connect, wait for the server to say “FDB 5.1.7”, then disconnect, fetch the appropriate fdb_c library file from foundationdb.org and connect and run as normal.


(John Brownlee) #5

We’ve added the libfdb_c files to the downloads page on foundationdb.org to help with setting up the multiversion client.


(Christophe Chevalier) #6

Great news :slight_smile:

Just so that I’m 100% sure I’m understanding properly: If I ship a binary that uses libfdb_c.5.1.7.dll, it should be able to connect to any 5.1.x cluster, but it will fail to connect to a 5.0.x or 5.2.x, is that right?

Looking at the Multi-Version API, it still look like we must have at least one compatible version of libfdb_c.dll loaded or in the folder, and after that it will automatically load new version if they are found.

But how can we boostrap this? Could there be a minimum libfdb_c.dll version that only supports querying the verison of a cluster, without any other API implemented? If we can use this bootstrap dll to get the version of the cluster, then download the proper libfdb_c file from somewhere, and implement the multi-version API, it could simplify the deployment of bindings a lot !


(A.J. Beamon) #7

That’s correct.

You must either use directly or load via the multi-version API a version of the library that is compatible with the cluster. We don’t currently support loading new libraries after the network is setup, so if you need to add support for a new version at present, then you’ll have to restart your client application.

The current way to manage this is more or less to just know what version your clusters are running and what they are about to upgrade to (possibly managed through some other service).

There is a actually already a protocol version exchange that happens when two processes connect, but this is slightly different than the release version (because multiple release versions can share the same protocol version, as mentioned above, and prerelease versions complicate this even more). And in fact, a cluster could consist of processes running multiple different compatible release versions. I think for the purposes you describe, though, knowing the version of one process would be good enough assuming that the process is truly part of the running cluster. Maybe it’s worth opening up a specific discussion about support for learning a cluster’s version from incompatible clients in order to solicit ideas.

One of the things this doesn’t help you with directly, though, is upgrades. To minimize downtime, you would want your clients to know what new version of the client to use before the upgrade happens, so you would have to have some other mechanism for making that information known.


New API to get the version of a running cluster without needing a matching libfdb_c library
(Christophe Chevalier) #8

I have created this topic New API to get the version of a running cluster without needing a matching libfdb_c library