FDB Simulate with fdbserver - documentation?

Hi all,

I wonder if anyone could point me in the right direction of some detailed documentation on FDB’s simulate feature? I’ve managed to run fdbserve with a few sample tests I downloaded, however I’m not very clear on what’s going on tbh, why I should use this, options, etc.
I’ve searched everywhere for some decent documentation but struggling to find any.

Any help would be very much appreciated.


I don’t think we have this well documented :frowning:

The only thing I have written is a short documentation about Client Simulation Testing - but depending on what you want to do, this might not be good enough. Though it should give you some very first idea how you could implement your own tests (though if you want to test something outside of a client you might want to use the internal API - for this you can use at any existing workload)

What is going on is actually quite simple:

The simulator simulates a cluster (potentially in different data centers) - it simulates disks, the network, and processes (and it also simulates time). It will then run normal FDB code.

At the start of a simulation it will initialize a random number generator with a seed (either it generates a true random number or you can pass a seed via the command line). Within that simulated cluster it will then run a set of workloads - described in a test-file. The simulator can then inject failures and some workloads will kill processes, machines, or even datacenters. A test is considered successful if no trace-lines with an error was printed.

If a test fails, you can look at the random seed and reproduce the failure as often as you want to.

This is just the gist of it. But it is actually quite easy to use. You can look at existing workloads to get an idea and at the existing test files to see how multiple workloads can be started (either concurrently or serial).

Though having better developer documentation is definitely something we should work on.

Thanks Markus - I shall have a play around.