We found that FoundationDB is a very good product and plan to use it. I have questions regarding how FoundationDB execute multiple getRange() requests.
- For example, if application sends 1000 getRange requests to FDB client and each getRange call is wrapped inside a CompletableFuture for
asynchronous invocation. Then how does FDB client handles these 1000 getRange requests? Let’s say FDB client might put these 1000 requests into a
request queue, after that how these requests are executed in parallel? I guess there are multiple threads are working on these pending requests,
I have two guesses and not sure which one is correct:
(1) Let’s just assume there is only 1 thread, Does this thread just get a getRange request from request queue, and immediately send to storage server, then get
the next request from requests queue and send to storage server? And asynchronously receive response from storage server. If we have multiple threads, then multiple threads are doing the same thing, right? If it is true, then there may have many getRange requests running in parallel which is good.
(2) I checked the getRange code in NativeAPI.actor.cpp, and looks like it does the sequential query from multiple storage servers if getRange needs to read data
from these servers because it wants to return the values in order, does this mean that each thread will get a getRange request from requests queue and then block on waiting for response from the storage server?
Then this thread will work on next one after finish this getRange request. If this is the case, suppose there are N working threads, then there are at most N getRange requests are running in parallel, right?
Can you please let me know FDB client is running like (1) or (2)? Basically how does FDB client execute getRange requests in parallel and it would be great if you can show me what is the correct running flow when FDB client receives a lot getRange requests.
Also does the connection channel between FDB client and storage server using something like bidirectional streaming? Or it is running like just send getKeyValues requests to storage server in parallel?
Do you think if client side batch multiple getKeyValues requests and send to storage server would be more efficient? I think even client has MultiGetRanges function, the latency should be same with executing getRange query in parallel since MultiGetRanges still needs to wait for all responses , just want to confirm.
Our current situation is that inside one read-only transaction, our application may send a lot of getRange requests to FDB client, say more than 2500, then there will be 2500 CompletableFutures, Do you think it would be a problem? Or if FDB already did well when running these getRange requests, then it would be no problem.
Thanks a lot and appreciate your reply.