Hi folks! I’m toying with a scheme to serve data derived from a read-only transaction from cache without re-deriving it or sacrificing linearizability. The scheme is roughly the following:
- Derive the data from a read-only transaction, and record the read snapshot and read conflict ranges of that transaction.
- Before serving from cache, validate the derived data by committing a transaction with the same read snapshot and read conflict ranges (and a dummy write conflict range to get the fdb client to actually send an rpc). If this commit is successful then you know that no transactions which could be known-committed before the request have changed the result of any of the reads involved in deriving the data.
A few questions:
- Does this seem sound?
- Waiting for the transaction batch to become durable on tlogs is this biggest source of latency for validating with this scheme, but since these validation transactions have no mutations it seems unnecessary for the commit proxy to wait for logging before responding. However, it’s not sound in general for commit proxies to reply to commit requests with no mutations before their commit version is durable, since a client could call getCommittedVersion and conclude incorrectly that mutations from any transactions with the same (or earlier) commit version are durable. Should we not expose a committed version for transactions with no mutations ?
- Has anyone tried something like this before?
Btw another source of transactions without mutations is the dummy transactions used in the native api to validate that transactions are not in-flight, so this could improve transaction latency in some failure scenarios.
Edit: wacky idea: what if we present the largest known committed version as the commit version for transactions without mutations? Not sure if we can present a sensible versionstamp though since we wouldn’t have a batch index.
We already do this for transactions with no mutations and no write conflict ranges: foundationdb/NativeAPI.actor.cpp at 8878de8c8f06ab7557b5edf40ffe6ae50a35ac19 · apple/foundationdb · GitHub ↩︎