I am using FoundationDB for my application. In my use case, the application frequently issues range queries that return thousands, even tens of thousands of key-value pairs.
I notice that there are two ways to execute a range query: 1) using tx.getRange().asList().get() to get all key-value pairs at once or 2) using tx.getRange().iterator() to get the key-value pairs iteratively. Below is my Java code:
private void execAsync(Database db, Range range, int expect) throws ExecutionException, InterruptedException {
Transaction tx = db.createTransaction();
AsyncIterator<KeyValue> iter = tx.getRange(range).iterator();
while (iter.hasNext()) {
KeyValue kv = iter.next();
kv.getKey();
kv.getValue();
}
tx.commit().get();
tx.close();
}
I run a microbenchmark program to measure the latencies of both ways. Below is the result (in milliseconds). My conclusion is asList() provides remarkably lower latency comparing with iterator(). The difference increases when the range size (the number of key-value pairs returned) increases.
Range Size Using asList() Using iterator()
1 1.213090 1.241620
10 1.237670 1.920940
20 1.299140 1.966210
50 1.386290 2.400080
100 1.515270 2.947490
1000 3.316320 6.013990
The results are gathered with 1 one node FDB and another node that runs my benchmark Java program. The database has 100k keys in total (each key is 8-byte length, each value is 100-byte length) and each query is run repeatedly 100k times.
My questions are:
Is it because asList() gets all key-value pairs in one call to FDB and iterator() uses multiple calls (equal to the number of fetched key-value pairs) in an async manner that causes the latency difference? If not, what is the explanation of the higher latency?
I want to go with iterator() for my application because with this way the application does not hold all key-value pairs in memory and it can process the data as soon as some key-value pairs are available. My major concern is the high latency of the current iterator(). Are there some tunings or configurations that reduce its latency? Ideally, I expect the latency of iterator() should not be much lower than asList() when the range size increases.
If I recall correctly, asList() issues the range request with option ‘WantAll’, which will prioritize fetching the data in as few server roundtrips as possible. This could explain the difference in time. However, asList() method would require you to have enough memory available to hold all the values in the requested range.
If I uses iterator() to retrieve N keys, the number of roundtrips to the server (not counting the request to get the read version) is N or less? Does iterator() also pre-fetch data? I think the latency should be reduced if iterator() also fetch the data in batch, let’s say, 10 keys at a time for them to be available locally. If it already does so, can this number (10) be configurable somehow?
The way that range queries work is that they request results in batches from the cluster using a byte limit determined by your streaming mode. The default streaming mode for the iterator is one that starts at a low byte limit and gradually increases (called ITERATOR), while calling asList uses WANT_ALL or EXACT, depending on whether you specify a row limit. If it has to do enough requests, the iterator mode would eventually reach the same byte limit as WANT_ALL, so a sufficiently large range should hopefully take a similar amount of time between the two queries in relative terms.
If you use the WANT_ALL streaming mode in your iterator example, I would expect the times to be similar.
I tried with the stream mode and you are right, the latency dropped when I used a different StreamMode than ITERATOR. Thank you very much!
Do you have more details on the differences between these stream modes? I traced the Java code but it seems that the parameter is passed to be processed in the C library. For example:
What is the fetch chunk size of each stream modes (SMALL, MEDIUM, LARGE, SERIAL, ITERATOR)?
What are the differences between LARGE and SERIAL?
If I select WANT_ALL, does it get all data in one call or does it still fetches data in chunks? If it still fetches data in chunks, is the chunk size larger than LARGE mode?
I had also filed Add to the Java docs a .asList() vs .iterator() comparison because you’re not the first person I’ve seen be confused by this and benchmark it. If you do enough playing around to have all the content that we should add to the docs on this subject anyway, it’d be wonderful for that to turn into a PR.
I guess the other thing to bench mark would be time to first byte. In theory, one should get better performance if one needs to, for example, pipeline asynchronous tasks (one per result back from the get range query) if one uses the iterator than if one uses asList.
Can we manual change the byte size instead of using STREAMING MODES? Let’s say instead of streaming mode=small, I want chunk size to be 9000 bytes. Can I do that?
