`fdb_transaction_get_range` API and `limit`

Development FoundationDB Layers
1

I recently encountered an API ergonomics issue with fdb_transaction_get_range and its use of limit = 0 to indicate unlimited.

I’m sharing the issue here for comments and hopefully as a heads-up to future binding/layer authors.

In RecordLayer, there is the notion of a Cursor and a Continuation, which are very useful abstractions to have when developing layers.

The most basic cursor is the KeyValueCursor. It is implemented as a FDB get range query and any key-value limit that the cursor might have would be translated directly as limits to the get range query.

Here is an example from RecordLayer Tests.

cursor = KeyValueCursor.Builder.withSubspace(subspace)
        .setContext(context)
        .setRange(TupleRange.allOf(Tuple.from(3)))
        .setContinuation(cursor.getNext().getContinuation().toBytes())
        .setScanProperties(new ScanProperties(ExecuteProperties.newBuilder().setReturnedRowLimit(3).build()))
        .build();

Cursor is designed to work across multiple transactions if a FoundationDB transaction limit (out-of-band limit) is encountered.

Should a situation arise where the limit number of key-values could not be retrieved in one transaction, we can build a new transaction and attempt to retrieve the remaining key-values by subtracting limit with the number of key-values already retrieved.

In the event we are not careful and build a cursor with a limit = 0, rather than receiving zero key-values, we will end up receiving key-values that we did not anticipate.

The reason for this is because get range considers limit = 0 to mean unlimited.

Here is my Rust test case that demonstrates the issue.

Would it make sense to call this out more explicitly in the C API documentation?

1 Like