Large Range Scans - avoid 5s limit

Using FoundationDB
10

Here is the almost complete code snippet that I currently use for this purpose. It should be straight forward to extend/update it:

// input byte[] start, byte[] end, boolean reverse
final AtomicReference<byte[]> seen = new AtomicReference<>(null);

fdb.readAsync(tx -> {
    // DO: Set timeout, num retries on the transaction object (tx)
    byte[] en = end;
    byte[] st = start;

    if (reverse) {
        en = seen.get() != null ? seen.get() : end;
    } else {
        // Note: nextKey is merged in master branch and not present on 6.2; copy it from github
        st = seen.get() != null ? ByteArrayUtil.nextKey(seen.get()) : start; 
    }

    return AsyncUtil.forEach(
            tx.getRange(st, en, ROW_LIMIT_UNLIMITED, reverse),
            kv -> {
                seen.set(kv.getKey());
                // DO: Consume the KV here...
            });
}, executor).join();
1 Like