Hi, is there a way to fetch “current” versionstamp using Java API?
I am trying to implement a layer to provide “tail” semantics on a key space:
key_space: log/pref_versionstamp -> {value}
Clients get two API: - getCurVersionStamp() - getUpdatesAfter(some_versionstamp)
Clients get an initial versionstamp bookmark using first API - in order to establish the initial read point. (they may read past K versions from this init-point in order to do some initialization).
They then loop and fetch newer versions using the second API by passing in existing bookmarked versionstamp, and updating the bookmark to the versionstamp of the last key fetched in a given iteration.
I understand that they can set watches to get similar results, but I am keeping that as a later optimization if required.
Why don’t you just start at the zero byte after your prefix for each log? There is an API for this in each binding, usually called KeyRange or something like that. It will read the first key and you can read forward from there. Then have each client write where it left off while processing and you can restart from where you left off in case the client fails.
I can think of two ways that might solve your problem.
Tail the last key in your log
This is the most direct way of solving your problem in that it gets you a for real version that someone has used to write a key into your log, which might be useful. You could do a snapshot read from the end of your log keyspace to get the last record and then parse the versionstamp out of it. Something like:
public CompletableFuture<Versionstamp> getCurVersionStamp(ReadTransaction tr) {
AsyncIterator<KeyValue> iterator = tr.getRange(logSubspace.range(), /* reverse = */ true, /* limit = */ 1).iterator();
return iterator.onHasNext(hasAny -> {
if (hasAny) {
// Get the last element from the log subspace and parse out the versionstamp
KeyValue kv = iterator.next();
return Tuple.fromBytes(kv.getKey()).getVersionstamp(0);
} else {
// Log subspace is empty
return null; // or a versionstamp of all zeroes if you prefer
}
});
}
If you do this in it’s own transaction or a read only transaction, this is fine. If you do it in a transaction that also performs writes, then you will conflict if there are any concurrent writes to the log subspace unless you pass in a snapshot transaction to the function outlined above.
Construct a versionstamp from your transaction's read version
This makes use of the fact that the first 8 bytes of a versionstamp are the commit version of the data associated with a record. Therefore, if you know the read version of the transaction, you also know that all data in your log subspace at version v will be prefixed by a version less than or equal to v and all future data added later will be prefixed with a version greater than v. So you can do something like:
This creates a new versionstamp prefixed by the current read version with all 0xff byte’s at the end. This guarantees that all versionstamps in the database will be less than or equal to (probably strictly less than) the constructed value, and all future versionstamps will be strictly greater than this value, which is I think what you want. Note that it doesn’t have to do any disk I/O and that it only depends on waiting for the read version (which is something that all FDB reads will have to do any way), though it somewhat depends on knowing the byte format used by FDB.
EDIT: I fixed some faulty method calls within the code snippets