Hi all. I’m new to FoundationDB, and have been trying to get a feel for what it is capable of.
I’ve set up a test environment which is doing about 1k reads/0.5k writes/s, running in ‘double ssd’ mode across 3 bare metal machines. Each machine is running 3 storage processes, and 1 log process, and each process has its own SSD. There’s a separate proxy process, and a separate stateless process on each machine.
I have enabled the trace file in the little client I’ve written using the Rust bindings. Latency98 seems to hover around the 20ms mark, but MaxRowReadLatency seems to jump around - sometimes <20ms, sometimes 500ms, occasionally exceeding a second. Over the course of around an hour, I see bucket counts like this:
I’m just wondering if that sort of maximum latency is normal, or whether it might be indicative of a problem somewhere?