Trying out redundancy mode 'double' on a single macOS machine

Using FoundationDB
1

Hi there,

I freshly started trying out foundationDB.

I set the locality_machineid for different processes to make fdb think that there are 3 machines.

However, after configure new single ssd the reported redundancy mode is unknown, like the following
fdb> status

Using cluster file `/usr/local/etc/foundationdb/fdb.cluster’.

Configuration:
Redundancy mode - unknown
Storage engine - ssd-2
Coordinators - 3

Cluster:
FoundationDB processes - 3
Machines - 3
Memory availability - 5.7 GB per process on machine with least available
Fault Tolerance - 0 machines
Server time - 05/08/18 00:34:41

Data:
Replication health - (Re)initializing automatic data distribution
Moving data - unknown (initializing)
Sum of key-value sizes - unknown
Disk space used - 0 MB

Operating space:
Storage server - 358.9 GB free on most full server
Log server - 358.9 GB free on most full server

Workload:
Read rate - 11 Hz
Write rate - 0 Hz
Transactions started - 0 Hz
Transactions committed - 0 Hz
Conflict rate - 0 Hz

Backup and DR:
Running backups - 0
Running DRs - 0

If I ran ‘configure double’ in this case, it hangs and reports the database is unavailable.

This is my config file
[general]
cluster_file = /usr/local/etc/foundationdb/fdb.cluster

[fdbserver]
command = /Users/me/psnl/3rdparty/apple/foundationdb/bin/fdbserver
public_address = 127.0.0.1:$ID
listen_address = 127.0.0.1:$ID
datadir = /Users/me/var/lib/foundationdb/data/$ID
logdir = /Users/me/var/log/foundationdb/

[fdbserver.9000]
locality_machineid = fdb9000
[fdbserver.9001]
locality_machineid = fdb9001
[fdbserver.9002]
locality_machineid = fdb9002

Did I miss configured anything ? THanks

2

My guess is that this was transient, and if you ran status again later it would report the correct redundancy mode once it had finished initializing data distribution. If you wait a while and it still doesn’t report the correct mode, then let me know.

This is an area where I think the documentation is lacking and the status output a bit misleading. If I remember correctly, I think that in order for the cluster to treat processes as being in different fault domains, you must set the locality_zoneid field on the process. It’s this field that the cluster uses to make sure that, for example, the transaction logs are in different fault domains.

3

Thanks for the reply.

For the first question, it still reports redundancy mode unknown after the replication health became health.
fdb> status

Using cluster file `/usr/local/etc/foundationdb/fdb.cluster’.

Configuration:
Redundancy mode - unknown
Storage engine - ssd-2
Coordinators - 3

Cluster:
FoundationDB processes - 3
Machines - 3
Memory availability - 7.7 GB per process on machine with least available
Fault Tolerance - 0 machines
Server time - 05/08/18 11:08:01

Data:
Replication health - Healthy
Moving data - 0.000 GB
Sum of key-value sizes - 0 MB
Disk space used - 0 MB

Operating space:
Storage server - 356.4 GB free on most full server
Log server - 356.4 GB free on most full server

Workload:
Read rate - 6 Hz
Write rate - 0 Hz
Transactions started - 2 Hz
Transactions committed - 0 Hz
Conflict rate - 0 Hz

Backup and DR:
Running backups - 0
Running DRs - 0

Client time: 05/08/18 11:08:01

fdb>

For the second question, setting zoneid works; but the redundancy mode still reports unknown. On that note, ‘status json’ can show the configuration as ‘double’ or ‘triple’.

fdb> status json

{
“client” : {
“cluster_file” : {
“path” : “/usr/local/etc/foundationdb/fdb.cluster”,
“up_to_date” : true
},
“coordinators” : {
“coordinators” : [
{
“address” : “127.0.0.1:9000”,
“reachable” : true
},
{
“address” : “127.0.0.1:9001”,
“reachable” : true
},
{
“address” : “127.0.0.1:9002”,
“reachable” : true
}
],
“quorum_reachable” : true
},
“database_status” : {
“available” : true,
“healthy” : true
},
“messages” : [
],
“timestamp” : 1525803448
},
“cluster” : {
“clients” : {
“count” : 0
},
“cluster_controller_timestamp” : 1525803448,
“configuration” : {
“coordinators_count” : 3,
“excluded_servers” : [
],
“redundancy_mode” : “triple”,
“remote_redundancy_mode” : “remote_none”,
“storage_engine” : “ssd-2”
},
… … …

4

Are you running off the master branch? It looks like there are some changes there which haven’t yet been fully reflected in the status output.

5

I think I found the issue, pull request created https://github.com/apple/foundationdb/pull/335

basically the name of the config field used to get the redundancy mode was inconsistent with what’s used in JSON.

6

Thanks! I’ve merged the PR.