I have a piece of code that was scanning the entire range from ‘’ to “\xFF” and db is loaded with my own test data. None of my keys are larger than 40 bytes. However, on a read of 10 million keys, I received 3 bad keys of size 1k to 5k bytes.
Unfortunately, I was testing with the head version - so it is hard to say what version it was referring to. I thought of setting up the environment again using the 6.3.9 release and running my Go client with the race detector turned on. Then I discovered this issue FDB go bindings fail go 1.14's unsafe pointer check · Issue #2843 · apple/foundationdb · GitHub - the 6.3.x release version also does not have the fix for that one. So I went back to attempting to build the latest master. I will update here with what I find.
However I wanted to clarify one thing before I dig deeper… On a range-scan using GetRange(), are their any special keys (like the status/json or something internal) that I should skip over?
It could very well be my client code and some race condition I am not accounting for.
If you are using the Directory Layer, or using another layer that uses the Directory Layer under the hood, then you could encounter some keys starting with the prefix \xFE with metadata about directories and partitions, but it is unlikely that these keys would be longer than a few hundred bytes.
For that reason I would probably use ‘\xFE’ as the end key, to stay clear of that.
If you want to make sure to have an isolated container for your keys, I would recommend that you add a short prefix to all your keys, if possible by using the Directory Layer (that has been designed for this exact reason), or by choosing your own prefix if you want to make it simpler.
Doing that, you’ll be able to have multiple datasets in the same cluster, and also make sure that no other layer (that also uses the Directory Layer) would “contaminate” your keyspace by mistake.
I wasn’t using any layer at all. It was a test db that I had added my own kv data. Since then I have recreated a new instance using master/head and I don’t see this issue anymore. So it may have been some data corruption. It was a ‘configure single ssd’ setup - so may be it had corrupted data.
I still couldn’t run some calls with go build -race on my app though… this is what I was hoping by remaining on master branch.