You’re right, but I guess the issue comes from how you would write this using using idiomatic .NET with async/await.
Let’s say I have a very simple WEB API controller, that create versionstamped keys but doesn’t need to know their value. This is nice and simple with the current API. note: I’m assuming that encoding of version stamps at the tuple layer is magical and just works. outside the scope of this sample
public class SomeApiController
{
#region Stuff..
// all initialized from the Web Application via HttpContext and DI...
private IFdbDatabase Db;
private CancellationToken Cancellation;
private IDynamicKeySubspace Location;
private string DoSomethingWithIt(Slice data) => "hello";
#endregion
// REST EndPoint
public async Task<SomeResult> SomeRestMethod(Guid id)
{
// here is the "business logic"
var data = await this.Db.ReadWriteAsync(async (tr) =>
{
// read a key
var val = await tr.GetAsync(this.Location.Keys.Encode("A", id));
// create stamped keys
tr.SetVersionStampedKey(this.Location.Keys.Encode("B", tr.CreateStamp(1)), Slice.FromString("some_value"));
tr.SetVersionStampedKey(this.Location.Keys.Encode("B", tr.CreateStamp(2)), Slice.FromString("some_other_value"));
return val;
}, this.Cancellation);
// return
return new SomeResult { Foo = DoSomethingWithIt(data) };
}
}
I would NOT want to write something like this:
await this.Db.ReadWriteAsync(async (tr) =>
{
// read a key
var val = await tr.GetAsync(this.Location.Keys.Encode("A", id));
// create stamped keys
tr.SetVersionStampedKey(this.Location.Keys.Encode("B", tr.CreateStamp(1)), Slice.FromString("some_value"));
tr.SetVersionStampedKey(this.Location.Keys.Encode("B", tr.CreateStamp(2)), Slice.FromString("some_other_value"));
tr.GetVersionStampAsync()
.ContinueWith((t) =>
{ // this runs somewhere on the ThreadPool, at any time in the future!
var stamp = t.Resut;
DoSomethingWithId(data, stamp);
}); // => if it fails, nobody will know about it!
);
}, this.Cancellation);
The Task continuation runs on another thread, maybe later, long after the HTTP context has been collected. And also it as no way to send it back to the client and could throw exceptions into nowhere
Now if I want to pass the actual versionstamp outside the scope of the retry loop and back to the controller while the HTTP context is still alive, I could try to change it like this, which is at least still using async/await:
// REST EndPoint
public async Task<SomeResult> SomeRestMethod(Guid id)
{
// first part of the business logic (that talks to the db)
(Slice data, Task<VersionStamp> stampTask) = await this.Db.ReadWriteAsync(async (tr) =>
{
// read a key
var val = await tr.GetAsync(this.Location.Keys.Encode("A", id));
// create stamped keys
tr.SetVersionStampedKey(this.Location.Keys.Encode("B", tr.CreateStamp(1)), Slice.FromString("some_value"));
tr.SetVersionStampedKey(this.Location.Keys.Encode("B", tr.CreateStamp(2)), Slice.FromString("some_other_value"));
return (val, tr.GetVersionStampAsync());
}, this.Cancellation);
// need another await
VersionStamp stamp = await stampTask; // <-- this is ugly!
var foo = DoSomethingWithIt(data, stamp); // hidden in here is the second part of the business logic
// return
return new SomeResult { Foo = foo };
}
But the actual business logic is split in two: the retry loop is doing the first part of the job, but then the actual “finish” is in this DoSomethingWithIt, which has to know that I used two stamps with user version 1 & 2. Also, the outer controller code has to act as the middle man, and also do the task resolving to get the stamp. => this is very tied to the implementation (using FoundationDB) and may not be easy to abstract away.
After playing a bit with it, I think the ideal would be to add an onSuccess handler on the retry loop logic, that gets called once the transction commits, and is passed the result of the inner handler, plus the resolved versionstamp. It can then consume and post-process the stamp.
// REST EndPoint
public async Task<SomeResult> SomeRestMethod(Guid id)
{
var foo = await this.Db.ReadWriteAsync(
handler: async (tr) =>
{ // this part runs inside the transaction, and can be retried multiple times
// read a key
var val = await tr.GetAsync(this.Location.Keys.Encode("A", id));
// create stamped keys
tr.SetVersionStampedKey(this.Location.Keys.Encode("B", tr.CreateStamp(1)), Slice.FromString("some_value"));
tr.SetVersionStampedKey(this.Location.Keys.Encode("B", tr.CreateStamp(2)), Slice.FromString("some_other_value"));
return val;
},
success: (val, stamp) =>
{ // this parts runs at most once, after the transaction has committed succesfully.
return DoSomethingWithIt(val, stamp);
},
ct: this.Cancellation
);
// no additional fdb-specific logic here!
return new SomeResult { Foo = foo };
}
Everything stamp relative is handled inside ReadWriteAsync(), and the result is the complete post-processed thing.
If I refactor this further, then no more fdb-logic is visible inside the controller itself
public class SomeApiController
{
#region Stuff..
private IFdbDatabase Db;
private CancellationToken Cancellation;
#endregion
// REST EndPoint
public async Task<SomeResult> SomeRestMethod(Guid id)
{
var engine = new MyBusinessLogicEngine(/*...*/);
var result = await engine.DealWithIt(this.Db, id, this.Cancellation);
// no additional fdb-specific logic here!
return new SomeResult { Foo = result };
}
}
// library in a different assembly somewhere
public class MyBusinessLogicEngine
{
private IDynamicKeySubspace Location;
public Task<string> DealWithIt(IFdbDatabase db, Guid id, CancellationToken ct)
{
return db.ReadWriteAsync(
handler: async (tr) =>
{ // this part runs inside the transaction, and can be retried multiple times
// read a key
var val = await tr.GetAsync(this.Location.Keys.Encode("A", id));
// create stamped keys
tr.SetVersionStampedKey(this.Location.Keys.Encode("B", tr.CreateStamp(1)), Slice.FromString("some_value"));
tr.SetVersionStampedKey(this.Location.Keys.Encode("B", tr.CreateStamp(2)), Slice.FromString("some_other_value"));
return val;
},
success: (val, stamp) =>
{ // this parts runs at most once, after the transaction has committed succesfully.
// second part of the business logic
return "hello:" + val + ":" + stamp;
},
ct: ct
);
}
}
But now, the method in the Business Logic class takes in a Database instance, and not a Transaction, so it cannot compose well with another layer that could read/write some keys at the same time.
I’ve seen this issue happening a lot in the last few years: the web controller has the db and is supposed to orchestrate everything in a single transaction to reduce latency. But all the various other libraries underneath will try open their own transactions in parallel. Sometimes by laziness, but sometimes by necessity (like a Blob Layer that has to upload more than 10 MB, or here deeply nested code that needs execute after the transaction completes, but before the controller gets back the result).
So by “not composing” well, I mean that now code inside retry loops has to take the responsibility of handling the lifetime of the transaction, and/or interleave part of its code back up the callstack, while keeping all its own scope (arguments, variables, context objects) it allocated alive long enough.