FoundationDB

Version length: Is 53 bits enough?

bindings

(Seph Gentle) #1

I’m going through cleaning up / rewriting the javascript bindings to bring them up to par.

There are a few places through which the user interacts with the version number. (Tuples, getVersionStamp, etc). In javascript the only number type is a double precision floating point, which can only safely hold integers up to 53 bits. Is this big enough to hold a version stamp (64 bits)? … Or should the JS binding just encode all version stamps into 8 byte binary buffers?


(Christophe Chevalier) #2

I think the default growth rate for versions is 1,000,000 / second, so 2^53 would be ~9 billion seconds, or about 285 years.

I know that you can change the “ticks per second” rate for the version in the configuration files, but I’m not sure why someone would more than 1E6 ticks/second?


(David Scherer) #3

Versions can advance somewhat faster through repeated recoveries, but I think it’s still unlikely to be a problem as things stand.

Not sure if it is worth the hassle of using a buffer or Uint32Array or whatever just to future proof against this issue.


(Alec Grieser) #4

Maybe I’m somewhat of a pessimist, but I actually wouldn’t feel too comfortable with only 285 years of headroom, especially if it’s not really 285 years because the version can advance fast due to recoveries, due to DR switchovers, or even due to clock drift. (In some theoretical future where we could commit more than 1 million transaction batches a second, we would need to advance at more than 1 million a second, though maybe that’s far off.)

Having an 8 byte version also means that now your version is a strict prefix of your 10 byte versionstamp (or 12 byte versionstamp), which may be a property you want, but maybe it doesn’t matter.

But it’s probably not really that important one way or the other. I suppose that if you ran into version value overflows, you could fix the return type (at a certain API version or higher) to something else.


Locking with FoundationDB
(A.J. Beamon) #5

I think our old node bindings did use 53 bit versions in some places, for what that’s worth (e.g. read versions).

On the other hand, it’s possible that in the future we find other reasons to accelerate versions advancing, and we’ll probably determine how safe that is based on the assumption that versions are a full 64 bits. As Alec says, you could change the API at that point to support 64 bit versions, though that wouldn’t be fun for the binding users.


(Christophe Chevalier) #6

Somewhat related to the original question: I was wondering if would be OK to reuse the highest bit (bit 63) of the transaction version stored in a versionstamp, as a flag to distinguish between incomplete and complete versionstamps.

This would allow me to have incomplete stamp using a random sequence of bytes (but with bit 63 forced to 1), that could then be recognized inside the key or value. This would simplifiy a lot the key encoders/decoders (Tuple Layer) which right now can only recognize FFFF....FFFFF has maybe an incomplete stamp.

This is only add the constraint that a version cannot be greater than 2^63-1 instead of 2^64-1 (so ~100 year lifespan instead of 200?)

Would this be an acceptable compromise?


(Seph Gentle) #7

After thinking about it I’m just going to make the node binding use buffers, with the number encoded big endian to allow lexical comparison to work properly.

I figure the only important interaction the client does with versions is copy them and sometimes compare them (a>b), so an 8 byte big endian buffer should be fine. And this is compatible with versionstamps. (And I think this will allow versions and version stamps to be compared natively too).