6.2.20 to 6.3.15 client upgrade questions

I have a Java application running on Ubuntu which is configured to connect to a primary FDB cluster (cluster A) and a secondary FDB cluster (cluster B). It attempts to connect to A on start-up and read the status JSON key to determine if the database is locked. If it is locked, it attempts to connect to B and read the status key JSON to determine if it is locked. This dance is to facilitate DR failover, and, any time a “database locked” error is encountered, this process starts again. An external process manages the failover, but that’s not really relevant to this issue.

I am attempting to upgrade the client version as given in the title. We are first upgrading the clients and setting API version 620 and we will deal with upgrading the servers later.

We receive this error after upgrading the clients.

Caused by: java.lang.RuntimeException: Failed to read cluster.database_locked or cluster.database_lock_state.locked from status={client={cluster_file={path=/tmp/foundation-db-cluster-files15548909453389961686/CLUSTER_NAME, up_to_date=true}, coordinators={coordinators=[{address=X.Y.Z.64:4500, reachable=false}, {address=X.Y.Z.84:4500, reachable=false}, {address=X.Y.Z.125:4500, reachable=false}, {address=X.Y.Z.126:4500, reachable=false}, {address=X.Y.Z.186:4500, reachable=false}], quorum_reachable=false}, database_status={available=false, healthy=false}, messages=[{description=Unable to reach a quorum of coordinators., name=quorum_not_reachable}], timestamp=1632757513}, cluster={layers={_valid=false}}}

The Failed to read cluster.database_locked or cluster.database_lock_state.locked is our error message, not something we’ve received directly from the database. The remainder is the original Exception from the database.

I’ve verified all the coordinators are available now and were available at that time. The cluster file has the same contents between both client versions.


As you can see, the IP addresses do match as X.Y.Z is a constant. The coordinators are definitely working, as the 6.2.20 clients still operate correctly.

Am I missing anything obvious here?

Do you have multi-version client setup on the upgraded boxes?

You will need to have 6.2.x libs available on the box when upgrading clients to 6.3.x. You can’t just set API version to 620 after upgrading. Essentially the client will have to make multiple connections to the cluster and whichever successfully connects will be used. This will work even when the cluster is upgraded later on.

See Using FoundationDB Clients — FoundationDB 6.3 for more details.