So I had an idea (probably not original) for how to write layers that interoperate across languages or across process / machine boundaries. This would be useful if someone had a highly-developed layer with a lot of functionality that you would like to use in the context of your transaction, but without needing to fork the layer.
It goes roughly like this:
- Layer One starts a transaction, gets a read version, and issues an RPC to Layer Two (e.g. another machine) containing the read version it just got.
- Layer Two starts a transaction and sets the read version to the read version from Layer One.
- Layer Two performs all operations normally, but doesn’t commit. It takes all the mutations and conflict ranges that would’ve been committed and sends them to Layer One, along with the higher-level result of the query/operation that has semantic meaning to Layer One (the real result of the RPC).
- Layer One examines the result of the RPC and decides to commit or abort the transaction as if it had performed all that work itself.
- In this example Layer One commits, so it applies all the mutations and conflict ranges from Layer Two. If it had aborted, nothing would’ve happened because Layer Two also never committed the transaction.
This probably relies on a feature somewhere that doesn’t exist yet around being able to examine the contents of the transaction’s mutations and conflict ranges, but I don’t think it does anything impossible.
I also don’t think it invalidates the workload management of FDB too much because the Layer One transaction will still wait in line for the read version if needed. Layer Two is doing work on behalf of Layer One that, under a monolithic layer model, it would’ve performed anyway. It is just happening somewhere else. It does have the potential to cause more wasted work if you spend too much time in the RPC transaction between One and Two and go over the 5s limit. It does add additional network traffic proportional to the size of the mutations and conflict ranges which would’ve only been sent by the client once which are now being sent by the Layer Two back to Layer One, then through the transaction pipeline.
This does rely on the layers trusting each other to both behave correctly (ignoring security, still not a security boundary), but I don’t think that’s a terribly large hurdle if you hide this functionality behind a wrapper library, as the results of the RPC and/or the side effects of it are the only thing the calling layer will need. The mutations and conflict ranges are an implementation detail.