How to get exact range of keys using fdb_transaction_get_range in C Programming

Using FoundationDB
1

Hi,
I have written a fdb client and storing different key-values. When I try to fetch particular range of keys, it is fetching all the keys which is not part that particular range.
For example,
I have stored following different types of keys,

Key1 : test1/test2/test3/1 Value : “TEST123-1”
Key2 : test1/test2/test3/2 Value : “TEST123-2”
Key3 : test1/test2/1 Value : “TEST12-1”
Key4 : test1/test2/2 Value : “TEST12-2”

I would like to fetch only keys with range of “test1/test2/test3”, But its fetching all for keys.

I’m using following options in get_range function,

int rowLimit = 1000;
int byteLimit = 1000;
int iteration = 1;
fdb_bool_t snapshot = 0;
fdb_bool_t reverse = 0;
FDBTransaction *tr;

fdb_database_create_transaction(db, &tr);

FDBFuture* f = fdb_transaction_get_range(tr, (const uint8_t *)“test1/test2/test3”, 0, 0, 1, (const uint8_t *)“\xff”, 1, 0, 1, rowLimit, byteLimit, FDB_STREAMING_MODE_WANT_ALL, iteration, snapshot, reverse);

Any change is needed in above code?

Thanks in advance.

2

The (exclusive) end key for test1/test2/test3 is test1/test2/test4, not \xff, which is the end of the (normal part of the) whole database.

1 Like
3

Thanks @MMcM. It’s working fine.
Is it possible to give same keys ranges in starting and end key values to get same range values like starting key as test1/test2/test3 and end key as “test1/test2/test3” ?

instead of giving starting key as “test1/test2/test3” and end key as “test1/test2/test4”.

But still I need some clarity on fetching data’s from FDB. How to give start and end keys for different range of values, for example,

KEY1 : test1/test2/test3/1
KEY2 : test1/test2/test3/2
KEY3 : test1/test2/test3/3

To fetch above key ranges , I have to give start key as “test1/test2/test3” and end key as “test1/test2/test4” and able to retrieve the keys.

If I have following keys as well,
KEY4 : test1/test2/1
KEY5 : test1/test2/2
KEY6 : test1/test2/3

If I give key range starts “test1/test2” and end key range “test1/test3”, it fetching all 6 keys values, instead of fetching KEY4 to KEY6.

Some more example keys likes,
KEY7: “test1/1”
KEY8: “test1/2”
KEY9: “test1/3”

I have to fetch only KEY 7 to KEY 9.

What is way to fetch unique key ranges ?

4

There are two things to keep in mind:

  1. Keys are lexicographically ordered unsigned byte strings. Your examples all show that.
  2. The end key is exclusive. I mentioned that last time.

It seems that the cases you are struggling with are constructing ranges for keys that have a common prefix. The higher level language bindings have get_range_startswith operations, but there is nothing magical about how they work.

Because the end key is exclusive, one needs to figure out the smallest key that is outside (greater than) the desired range. Again, the higher level language bindings have functions to do this. But the basic idea is to increment the last byte. That’s why '3' → '4' in your original example. There remains the case where the last byte was \xFF, in which case you lop that off and “carry” the increment over to the previous byte.

Another refinement would be wanting the strict children: that is, excluding the first key. Then one needs the smallest key greater than the parent key. Which is gotten by appending \x00.

Now, to your new examples. You want test1/test2/1 through test1/test2/3, but not test1/test2/test3/1. You have to figure out where the ones you want stop. For example, maybe you want just the ones with digits. Then your end key is the first non-digit, which is ':'.

I am by no means certain that is what you want, however. In particular, because you didn’t give your 9 keys in order. That would be:

test1/1
test1/2
test1/3
test1/test2/1
test1/test2/2
test1/test2/3
test1/test2/test3/1
test1/test2/test3/2
test1/test2/test3/3

So, perhaps the problem really is that you need to think about a way of encoding keys that arranges for contiguous ranges in lexicographical order to match what you need to fetch.

1 Like
5

Thanks @MMcM for detailed explanation. I got the flow and now its working for me.