Some questions about cluster configuration and test

Using FoundationDB
1

I have build a cluster in AWS with following machines and configurations
M1: m5.4xlarge (16 vcpu + 64GB DRAM)
M2 & M3: m5.2xlarge (8 vcpu + 32 GB DRAM)
I use memory storage engine.
For M1 I set 16 processes with class=storage ,memory = 10GiB, storage_memory = 4GiB and cache_memory = 1GiB.
For M2 I set 8 processes with class=transaction.
For M3 I set 8 processes with class=stateless.

My first question is that if I configure the class of process and memory properly.

My second question is that I found there is little operating space left after I create a database. The status result is shown following. I want to know what is operating spaces and how to expand it.

Configuration:
  Redundancy mode        - single
  Storage engine         - memory-2
  Coordinators           - 3
  Usable Regions         - 1

Cluster:
  FoundationDB processes - 32
  Zones                  - 3
  Machines               - 3
  Memory availability    - 4.1 GB per process on machine with least available
  Fault Tolerance        - 0 machines
  Server time            - 04/08/22 14:08:23

Data:
  Replication health     - Healthy
  Moving data            - 0.000 GB
  Sum of key-value sizes - 0 MB
  Disk space used        - 316 MB

Operating space:
  Storage server         - 0.8 GB free on most full server
  Log server             - 3.2 GB free on most full server

Workload:
  Read rate              - 37 Hz
  Write rate             - 0 Hz
  Transactions started   - 10 Hz
  Transactions committed - 0 Hz
  Conflict rate          - 0 Hz

Backup and DR:
  Running backups        - 0
  Running DRs            - 0

The third question is that when I try to run a multitest, I need to set the class of some processes as test.
The docs said that these processes will act as clients.
So how many test class processes should I set to get a better test result (higher throughput).

Look forward to anyone’s help.
Thanks very much!

2

My second question

For the memory storage engine the operating space is set with the parameter storage_memory. The default value is 1GiB.

See more details at Configuration — FoundationDB 7.1

storage_memory = 4GiB and cache_memory = 1GiB .

Seems you have set storage_memory = 1GiB and cache_memory = 4GiB .

3

If you specify memory = 10Gib then 16 storage processes will require up to 160 Gib but you have only 64Gib of RAM. This configuration is not stable.

But having 6 Gib above the cache_memory for storage processes are extra. Most of additional memory is required for universal processes, but not for dedicated storage processes.

So if you have cache_memory = 4 Gib, then memory = 6 Gib is enouth for storage processes. So you need to have 16*6Gib + 2 Gib = 98 Gib memory for M1 machines

You haven’t written how many machines of each type do you have. If you have only 3 machines, one machine of each type, then placing all process classes on one machine is not good from the availability point of view: your cluster can not continue after failing of any machine.

Another point is one transaction process can handly about 8 storage processes. So if you have 16 storage processes, then 2 transaction processes would be enough.

So I’d recommend you

  • To have 3 similar machines with 8 cpu and 64 GB RAM

    1. one stateless process
    2. one transaction process
    3. 8 storage processes

    memory = 6Gib
    cache_memory = 4Gib
    storage_memory = 4Gib

  • or to have 15 small machines (2 CPU, 14 GB RAM)

    1. Three of them with one stateless process and one transaction process
    2. 12 of them with two storage processes each

    the configuration is the same:
    memory = 6Gib
    cache_memory = 4Gib
    storage_memory = 4Gib

1 Like
4

Thanks for your patient reply! It helps me a lot.