Benchmark output

Using FoundationDB
1

Hi,

I am running foundationdb synthetic random read write benchmark, I am having some trouble in understanding the results of the benchmark. The result is as follows for one of the client processes.

startingConfiguration: start
setting up test (RandomReadWriteTest)…
running test (RandomReadWriteTest)…
RandomReadWriteTest complete
checking test (RandomReadWriteTest)…
fetching metrics (RandomReadWriteTest)…
Metric (0, 0): Measured Duration, 300.000000, 300
Metric (0, 1): Transactions/sec, 341.896667, 342
Metric (0, 2): Operations/sec, 3452.866667, 3.45e+03
Metric (0, 3): A Transactions, 92399.000000, 92399
Metric (0, 4): B Transactions, 10170.000000, 10170
Metric (0, 5): Retries, 523.000000, 523
Metric (0, 6): Mean load time (seconds), 2.653203, 2.65
Metric (0, 7): Read rows, 101700.000000, 1.02e+05
Metric (0, 8): Write rows, 934160.000000, 9.34e+05
Metric (0, 9): Mean Latency (ms), 579.081821, 579
Metric (0, 10): Median Latency (ms, averaged), 475.733280, 476
Metric (0, 11): 90% Latency (ms, averaged), 1270.120382, 1.27e+03
Metric (0, 12): 98% Latency (ms, averaged), 1633.173466, 1.63e+03
Metric (0, 13): Max Latency (ms, averaged), 4480.281830, 4.48e+03
Metric (0, 14): Mean Row Read Latency (ms), 1.397406, 1.4
Metric (0, 15): Median Row Read Latency (ms, averaged), 1.105785, 1.11
Metric (0, 16): Max Row Read Latency (ms, averaged), 123.408794, 123
Metric (0, 17): Mean Total Read Latency (ms), 0.355296, 0.355
Metric (0, 18): Median Total Read Latency (ms, averaged), 0.000000, 0
Metric (0, 19): Max Total Latency (ms, averaged), 123.408794, 123
Metric (0, 20): Mean GRV Latency (ms), 561.424525, 561
Metric (0, 21): Median GRV Latency (ms, averaged), 458.384752, 458
Metric (0, 22): Max GRV Latency (ms, averaged), 1939.560652, 1.94e+03
Metric (0, 23): Mean Commit Latency (ms), 6.001462, 6
Metric (0, 24): Median Commit Latency (ms, averaged), 3.398657, 3.4
Metric (0, 25): Max Commit Latency (ms, averaged), 149.589062, 150
Metric (0, 26): Read rows/sec, 339.000000, 339
Metric (0, 27): Write rows/sec, 3113.866667, 3.11e+03
Metric (0, 28): Bytes read/sec, 178992.000000, 1.79e+05
Metric (0, 29): Bytes written/sec, 1644121.600000, 1.64e+06

I want to know that what does Metric (0, 9): Mean Latency (ms), 808.910736, 809 represent here?, The mean total read latency and mean total commit latency seems to be close to be fine, but the mean latency is more than 0.5 seconds which is quite a lot. could you kindly tell how is the mean latency calculated in the benchmark? :slight_smile:

2

The code behind this is relatively straight forward, so you’re welcome to take a look: foundationdb/ReadWrite.actor.cpp at e7d7b39f12afa8ea21aeb94972fc79cd7b5edbab · apple/foundationdb · GitHub

The latencies without any description are the total transaction latency.

If I read between the lines in this result, it looks like you’ve likely saturated the underlying FDB database. Your GRV latency getting into the seconds probably suggests that ratekeeper is throttling the cluster, so I’d take this more as a throughput at saturation test than anything about the latency of FDB.

As you’re doing 342 transactions/second, I’ll assume that this means that you’re running this locally, and the result probably makes sense then. I’d expect it to be pretty easy to saturate a local, one process FDB cluster.

3

As you’re doing 342 transactions/second, I’ll assume that this means that you’re running this locally, and the result probably makes sense then

But each transaction has 10 operations

4

Thank you for your response. You are correct that the following is a benchmark saturation test, but I am running the benchmark on a cluster with two server machines and 3 client machines on AWS. The 3 client machines have total of 128 client processes that are making transaction to the two server machines. This is the output for one of the client processes (from total of 128 client processes).
I wanted to know, what might be the reason for such a high value of mean latency even if the test is for throughput saturation.

Thanks

5

On saturation usually latency depends on the number of simultaneous transactions. If you decrease this number you decrease latency as well, but the throutput will not change significally.

6

Generally I like to do tests like this by stepping up the requests per second over time to understand when you hit saturation. There is no doubt in my mind that 3 machines will saturate a small FDB cluster running at full speed.