We had to recover a cluster last night from a backup with fdbrestore. During the restore process we started with a replication factor of double and manually set 3 coordinators on the fdbcli. After recovery we moved to triple replication and attempted to set coordinators auto. Unfortunately we’re getting the following response from fdbcli.
fdb> configure auto Unable to provide advice for the current configuration.
After digging into a bit of the code it looks like this is occurs in parseConfig in https://github.com/apple/foundationdb/blob/6.0.18/fdbclient/ManagementAPI.actor.cpp#L462
It appears this code parses the machine readable configuration (json?) and if any one of a number attributes are missing or not what it expects it returns an empty ConfigureAutoResult(). Where all values of the ConfigureAutoResult are just set to -1 and the isValid() call returns false https://github.com/apple/foundationdb/blob/6.0.18/fdbclient/ManagementAPI.h#L98-L105. This results in the
Unable to provide advice for the current configuration. here https://github.com/apple/foundationdb/blob/6.0.18/fdbcli/fdbcli.actor.cpp#L1523
We’re hoping someone can help answer the following.
1.) What’s the best way to debug what config properties that are missing or unexpected and could be causing the parseConfig to return the invalid ConfigureAutoResult.
2.) If we can’t resolve issue #1 is it safe for us to just go ahead and manually set coordinators again with the existing 3 coordinators and add 2 more so we can get to 5 coordinators and a Fault Tolerance of 2 machines.
Cluster Info ---------------- FDB Version 6.0.18 Configuration: Redundancy mode - triple Storage engine - ssd-2 Coordinators - 3 Desired Proxies - 15 Desired Logs - 45 Fault Tolerance - 1 machine
OS --------------------------------------- Description: Ubuntu 16.04.6 LTS Release: 16.04 Codename: xenial Linux version 4.4.0-1050-aws (buildd@lgw01-amd64-017) (gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.5) ) #59-Ubuntu SMP Tue Jan 30 19:57:10 UTC 2018