Client library backwards compatibility issues

Hey there -

I updated my client libraries from 5.2 to 6.0 but now I can’t seem to communicate with my cluster. Is this expected behavior? I’m able to connect to the cluster on any other client that’s using the 5.2 library.

$ fdbcli
Using cluster file `fdb.cluster'.

WARNING: One or more of the processes in the cluster is incompatible with this version of fdbcli.

The database is unavailable; type `status' for more information.

Welcome to the fdbcli. For help, type `help'.
fdb> status

WARNING: One or more of the processes in the cluster is incompatible with this version of fdbcli.

Using cluster file `fdb.cluster'.

Could not communicate with a quorum of coordination servers:
  10.49.38.10:4500  (unreachable)
  10.49.38.23:4500  (unreachable)
  10.49.38.36:4500  (unreachable)
  10.49.38.52:4500  (unreachable)
  10.49.38.72:4500  (unreachable)

That’s expected. For a given FDB version major.minor.patch, only .patch-level releases are protocol compatible. This means any major or minor changes to the server imply a new protocol across the cluster. The fdb_c library uses this same protocol to talk to the cluster, so it impacts all clients, as well.

(Note that protocol changes are not the same as storage engine/format changes, as you might have noticed – the raw data on disk is generally stable across versions.)

The correct thing to do here is to use the multi-version client support, documented here. The TL;DR is you need to put copies of libfdb_c-5.2.so and libfdb_c-6.0.so from the client library packages into a directory, and point your client applications to that directory during startup. When they try to talk to the cluster, they will negotiate a protocol version, then transparently load the right library and use that to communicate.

First, you would want to deploy a new version of all your client applications/layers with the multi-version API configured, and both 5.2 and 6.0 libraries available somewhere. You do not want to make any client-level changes, just a redeploy (perhaps with a few extra command line options pointing to the multi-version directory.) Then, you can roll out the new cluster.


There is also the question of the API version you request in your client application. The thing is, you want to keep requesting the old API version with the new cluster until it’s fully rolled out. For instance, you would be done deploying the 6.0 cluster, but you’d still request an API version of 520. Later on you can update the API version as part of routine maintenance in your client applications, once everything is stable. You always want to do this last.

The reason for this is simple but subtle and took me a while to boil down easily: features are not the same as protocol-incompatible changes. Even though you need a different protocol for 5.2 and 6.0, 6.0 still supports the same features as 5.2. It can still satisfy the constraint you “want all 5.2.x based features”. The underlying protocol/dynamically loaded client library is orthogonal to this. This means you could also still downgrade to 5.2 as well (I think.) But once you request an API version of 600, there’s no going back.


Note: As far as I know, fdbcli itself does not support the multi-client library feature, because it is not a client of libfdb_c.so at all – it shares code and features directly with the server codebase, so it’s much more tightly coupled (for example, what should happen when you use new 6.0 multi-region support in fdbcli, while talking to fdb 5.2?) It’s unclear if this can or should ever change – so, in the mean time you probably also want to keep multiple versions of fdbcli around, too.

1 Like