Range-reading all key-values

Using FoundationDB
1

I’m trying to range-read all the key-values in my database for debugging purposes. I’m doing this using the Golang client. Though things don’t work as I’d expect them to.

There are two kinds of keys stored in my database:

  1. Versionstamped keys using the method Transaction.SetVersionstampedKey(Key, []byte)
  2. Tuples containing two strings: Tuple{"prefix", "unique name"}

I have found ways to range-read each of these kinds of keys, but I haven’t found a single range that reads them all.

To read the versionstamped keys, I can use this range:

rng := SelectorRange{
  Begin: FirstGreaterOrEqual(Key("")),
  End:   LastLessOrEqual(Key([]byte{0xff})),
}

To read the tuple keys, I use this range:

rng, _ := PrefixRange(Tuple{"prefix"}.Pack())

I understand why the second range only reads the tuple keys. What doesn’t make sense to me is why the first range only reads the versionstamped keys. I expected the first range to read all the keys, including the tuple keys. Can someone enlighten me?

2

The one issue I see with the top query is with the end selector. The end of a range is exclusive of the key that the selector resolves to, and in this case it’s going to resolve to the last key in normal key space. If you only have one tuple and it comes after the versionstamped key, then that could explain why it’s missing. If you change your range to the following, it will include all of the keys in the database (excluding system keys):

rng := SelectorRange{
  Begin: FirstGreaterOrEqual(Key("")),
  End:   FirstGreaterOrEqual(Key([]byte{0xff})),
}

This change only fixes the ability to read the last key, though, so if you have multiple tuple keys and you aren’t seeing any, it won’t fully explain your issue.

Another possible thing that could be happening is that there is too much data to read in the database in one transaction. If you do try to read the entire database in one transaction and it takes too long, you will eventually get back an error. Depending on how that error is being handled, it could appear that you’ve completed the range read but are missing data at the end.

Let me know if these suggestions don’t seem to fix the issue. I’m not immediately spotting any other problems, but I can help diagnose if it’s still not working.

1 Like
3

You were correct. The ending was the problem. Thanks.