How to achieve (1000 tx 1000 reads each)/sec on single client

Assuming when you say “instance” you mean once process launched (for example, node <some_program> then each instance would be using a different instance of the FDB library and therefore a network thread. This is what you would want to happen, combining requests from different client app instances on the same box would not be an improvement.

An FDB network thread only opens at most one TCP connection to any cluster process it communicates with, and all requests to that node share that one connection. The FDB client talks to several roles in the cluster, but mainly the proxies (for getting read versions, discovering what storage server to talk to for a key range, and submitting transactions) and the storage servers.

If your values are 500 bytes each, and this is the amount of requests you need to perform in one second, then you are trying to read 500 MB/s with 1 million requests per second from a single instance of the FDB client library. While your cluster can go this fast, a single instance of the FDB client library cannot.

It also seems likely that your network connection could not go this fast. Although 500 MBytes/s sounds slower than 10 Gigabit ethernet (which is 1.1GBytes / second) this traffic pattern of 1 million small requests per second to many different processes is very different from, say, streaming a file at 500 MByte/s.

Supposing though that your network connection was able to handle this, and your client machine was powerful enough, there may be some future in which this speed from a single client application is possible. There is a plan to eventually (no idea when…) have an RPC layer in front of FoundationDB.

This could help your use case because then a single process application could perform its reads with many threads and many connections to many RPC service instances in the FDB cluster. This would remove the bottleneck of all requests being routed through the FDB client library instance’s single network thread.

There is a discussion of the RPC layer concept here:

1 Like