I may have an idea what’s happening here. fdb_cluster_create_database does the actual database creation on the network thread, and then signals completion via a future. It’s holding onto a couple references to the database object (which in turn holds a reference to another object, say dbContext, which is shared with the transaction). When the job is done and it signals the future, the C code is immediately getting that database object, using it to create a transaction, and then destroying it.
If the above completes in the following order, then I think we run into problems:
- Future is signaled
- Client code runs at least through
fdb_database_destroy - The task that signaled the future finishes wrapping up and drops its references to the database object.
The expectation was that 3 completes before 2, in which case the call to fdb_database_destroy schedules the last decrement of the database object, causing the dbContext reference count to be decremented after the transaction increments it.
In the case where 2 happens before 3, then the last reference to the database object is deleted on the network thread. It schedules the decrement of the dbContext reference count, but as I mentioned in a previous message, the fact that this happens on the network thread means that it can jump in line in front of the transaction work that increments the reference count. As a result, dbContext is deleted prematurely.
This is a good find, thanks for the reproduction. I’ll file an issue in GitHub to address it.
EDIT: https://github.com/apple/foundationdb/issues/518
As a side note, the reason we probably haven’t seen this previously is that we don’t typically delete databases so quickly after creating them. The recommended practice is to keep the same database object around for as long as you’ll want to be creating transactions and then close it only when you’re done interacting with the database.