Need help to understand setting fdbserver.ID

Using FoundationDB
1

I was reading following recommendation on number of processes and I am not clear how to spread these many processes on 3 nodes using configuration file or “configuration” command or combination of both. Can someone please help to understand the setup needed.

" The recommended minimum number of class=transaction (log server) processes is 8 (active) + 2 (standby) and the recommended minimum number for class=stateless processes is 1 (GRV proxy) + 3 (commit proxy) + 1 (resolver) + 1 (cluster controller) + 1 (master) + 2 (standby). It is better to spread the transaction and stateless processes across as many machines as possible."

So far I have used following configuration file and “configuration command” to setup 2 storage fdbserver.ID process per node , 3 coordinator , double redundancy as follows. But i am not clear should i use fdbserver.ID (and use class=storage/transaction/stateless) or “configuration” command to setup the recommended processes and use all the 4 cpu per node i have for a given EC2 server i am using.

I also dont know how to check current count for these processes.

[fdbserver.4500]
datadir = /data/foundationdb/data1

[fdbserver.4501]
datadir = /data/foundationdb/data2

========
fdb> status details

Using cluster file `/etc/foundationdb/fdb.cluster’.

Configuration:
Redundancy mode - double
Storage engine - ssd-2
Coordinators - 3
Usable Regions - 1

Cluster:
FoundationDB processes - 6
Zones - 3
Machines - 3
Memory availability - 8.0 GB per process on machine with least available
Fault Tolerance - 1 machines
Server time - 05/27/22 22:45:03

Data:
Replication health - Healthy
Moving data - 0.000 GB
Sum of key-value sizes - 0 MB
Disk space used - 944 MB

Operating space:
Storage server - 2034.1 GB free on most full server
Log server - 2034.1 GB free on most full server

Workload:
Read rate - 20 Hz
Write rate - 0 Hz
Transactions started - 5 Hz
Transactions committed - 0 Hz
Conflict rate - 0 Hz

Backup and DR:
Running backups - 0
Running DRs - 0

Process performance details:

  •  (  1% cpu;  2% machine; 0.000 Gbps;  0% disk IO; 0.4 GB / 8.0 GB RAM  )

  • (  1% cpu;  2% machine; 0.000 Gbps;  0% disk IO; 0.4 GB / 8.0 GB RAM  )

  •  (  1% cpu;  2% machine; 0.001 Gbps;  0% disk IO; 0.4 GB / 8.0 GB RAM  )

  • (  1% cpu;  2% machine; 0.001 Gbps;  0% disk IO; 0.3 GB / 8.0 GB RAM  )

  •  (  2% cpu;  2% machine; 0.001 Gbps;  0% disk IO; 0.4 GB / 8.0 GB RAM  )

  •  (  0% cpu;  2% machine; 0.001 Gbps;  0% disk IO; 0.4 GB / 8.0 GB RAM  )

Coordination servers:
*:4500 (reachable)
*:4500 (reachable)
*:4500 (reachable)

2

For setting process class, either editing Configuration — FoundationDB 7.1 to set the class as an argument to fdbserver, or using the setclass command in fdbcli should be fine.

foundationdb should recruit a reasonable topology without setting process class at all. IMO the big thing to get right is that if you have extra processes in the cluster that don’t have dedicated disks, you should give them the stateless process class so that you don’t accidentally end up using the root volume for a storage server if you intend for all your disks to be on EBS for example.

" The recommended minimum number of class=transaction (log server) processes is 8 (active) + 2 (standby) and the recommended minimum number for class=stateless processes is 1 (GRV proxy) + 3 (commit proxy) + 1 (resolver) + 1 (cluster controller) + 1 (master) + 2 (standby). It is better to spread the transaction and stateless processes across as many machines as possible."

I think this advice doesn’t really apply until you have a much larger cluster with triple replication, and at least 10 disks you want to use specifically for transaction logs.

3

Thanks for the reply. Just want to confirm I got this right (sorry to bug you but i really appreciate the help…) .It will be nice if sample configuration file for real production small and large cluster is shared here for reference.

I am setting 3 AZ server cluster with dedicated /data EBS volume attached and each node has 4 CPU and 32 GB RAM.

So far I have set datadir = /data/foundationdb/data and logdir = /data/foundationdb/log and then started the cluster and set the storage engine to ssd, number of co-ordinator =3 and redundancy = double using fdbcli/configuration commands. As per document and your reply it looks like I don’t need to set the class or number of processes for a small cluster of 3 nodes. Correct me if wrong please

**I need help to understand this QQ ** I have total 12 CPU so how FoundationDB take advantage of all the 4 CPU on the given node if I do not set any process class and the number of processes. Following note says the default values for various process. But let’s say I want to set more logs process or commit processes how I decide the number if I have total 12 cpu to work with. Does FoundationDB need one process per CPU or you can have more number of processes than the number of CPU i.1. 12.

Also if I want to add more storage should i simply use on /data and increase the volume or it is better to have multiple disk for example /data1 /data2 etc and use FDBserver.ID to assign class=storage for them.

NOTE:-the present release, the default value for commit proxies and log servers is 3 and for GRV proxies and resolvers is 1. You should not set the value of a process type to less than its default.

process types

A FoundationDB cluster employs server processes of different types. It automatically allocates these processes in default numbers appropriate for small-to-medium sized clusters.

For large clusters, you can manually set the allocated number of processes of a given type. Valid process types are:

*** grv_proxies**
*** commit_proxies**
*** resolvers**
*** logs**

Set the process using configure [grv_proxies|commit_proxies|resolvers|logs]=<N> , where <N> is an integer greater than 0, or -1 to reset the value to its default.

For recommendations on appropriate values for process types in large clusters, see Guidelines for setting process class.