FDBDataBase Usage from Java API

Using FoundationDB
1

Hi,

I ran into a few issues while working with Java API. Could someone please check what I could be doing wrong here:

  1. Database.run() method accepts an executor and documents

    “e the {@link Executor} to use for asynchronous callbacks”.

    However, FDBDataBase.run() method seems to not use it:

public <T> T run(Function<? super Transaction, T> retryable, Executor e) {
     Transaction t = this.createTransaction();

Should the this.createTransaction() be taking in the passed executor in the constructor?

  1. Deadlock with FDBDatabased.readAsync()

I have this trivial code snippet to set a key in DB with a unique id - I am using VersionStamp to derive the unique id and also want to return it to the caller in same call:

CompletableFuture<byte[]> create(final String key) {
    final byte[] b = new Tuple().add(key).pack();
    return db.runAsync((Function<Transaction, CompletableFuture<byte[]>>) tx -> {
        tx.mutate(MutationType.SET_VERSIONSTAMPED_VALUE, b, Tuple.from(Versionstamp.incomplete()).packWithVersionstamp());
        return tx.getVersionstamp();
    });
}

It appears that runAsync() method is composing the return Future from my lambda function and chaining this Future with commit() post the future completion. Since this future cannot complete till commit() is called (future is fetching VersionStamp value), this leads to a deadlock.

 return AsyncUtil.composeHandleAsync(process.thenComposeAsync(returnVal ->
		trRef.get().commit().thenApply(o -> {

Am I using runAsync() method incorrectly here?

–
thanks,
gaurav

2

With regard to question 1, yeah, that’s a bug. The createTransaction parameter should pass along the executor given. Instead, I believe that this will use the executor associated with the database. Oops. Feel free to file this as an issue on GitHub.

With regard to question 2, that is essentially expected behavior. The runAsync retry loop will only call commit after the returned future has completed (so you could do something like read a key from database, make some write based on your read, and then commit). Instead, you need to do something like:

byte[] versionstamp = db.run(tx -> {
    // some operation with your transaction
    return tx.getVersionstamp();
}).join();

This will synchronously commit your transaction, though, so you could instead do:

CompletableFuture<byte[]> versionstampFuture = db.runAsync(tx -> {
   // some operation with your transaction
   return CompletableFuture.completedFuture(tx.getVersionstamp());
}).thenCompose(Function.identity());

What this does is return a future of a future of the versionstamp within your loop. The retry loop will wait for that future to complete* and then call commit. Because the future is already done, it will call commit essentially instantaneously after returning. Once the commit is done, the final callback (the thenCompose there) will unwrap the versionstamp from the returned future. It’s a little bit awkward, but I think it would work.

*Or, rather, set a callback for that future’s completion

3

Appreciate the response. I will file the bug/pr for (1).

4

@alloc Should we default to making all our write calls asynchronous by using runAsync()?

If yes, I’m not exactly sure how this could be done since (as stated above) all the write methods are synchronous. Therefore, I’m guessing FDB wants me to use synchronous writes and asynchronous reads? Am I thinking through this correctly?

5

You would be correct. Writes only write into a client-local cache, so there’s nothing that can be blocked on there. The “real” write operation is commit(), which is async.

6

Could I call commit() myself and return the CompletableFuture<Void> so that the calling code can know exactly when the write completes? For instance:

public CompletableFuture<Void> writeStuff() {
  return database.runAsync((tr) -> {
    tr.set(key, value);
    return tr.commit();
  });
}
7

If your calling code ever only does one write per transaction, then yes.

After you call commit(), you aren’t allowed to do anything else with your transaction, as it is now completed. The intended model is that after your calling code finishes all of its writes, you then call commit, and use the future returned from commit to know if/when the transaction committed (and all values set() in the database were written), or if the transaction failed.

8

I don’t think you can do this because runAsync also calls commit implicitly when you return successfully from the passed function, and I believe it’s invalid to commit twice. However, you will be able to know when the commit completes by waiting on success from the future returned by runAsync.

9

I tried doing this, but I end up with a NullPointerException, presumably because I am returning null in my Retryable:

db.runAsync((tr) -> {
  tr.set(key, value);
  return null;
}).join(); // throws NullPointerException
10

runAsync expects the passed in function to return a CompletableFuture, and it waits for that future to complete before committing. For example, the following should work if you just want to return null:

db.runAsync((tr) -> {
  tr.set(key, value);
  return CompletableFuture.completedFuture((Void)null);
}).join();

If you intend to just join the result immediately, though, you can do something equivalent with db.run:

db.run((tr) -> {
  tr.set(key, value);
  return null;
});
1 Like