Question regarding WATCH future

Development
1

I am currently working on UNIT_TESTS instruction for Rust binding tester.

In the tests for watches I noticed the following code in Java and Go bindings.

List<CompletableFuture<Void>> watches = db.run(tr -> {
	List<CompletableFuture<Void>> watchList = new LinkedList<>();
	watchList.add(tr.watch("foo".getBytes()));
	watchList.add(tr.watch("bar".getBytes()));

	tr.set("foo".getBytes(), "f".getBytes());
	return watchList;
});

_, e = db.Transact(func(tr fdb.Transaction) (interface{}, error) {
	watches[0] = tr.Watch(fdb.Key("w0"))
	watches[1] = tr.Watch(fdb.Key("w1"))
	watches[2] = tr.Watch(fdb.Key("w2"))
	watches[3] = tr.Watch(fdb.Key("w3"))

	tr.Set(fdb.Key("w0"), []byte("0"))
	tr.Clear(fdb.Key("w1"))
	return nil, nil
})

In the above code, we seems to be leaking FDB future from the function closure passed to db.run and db.Transact.

Since lifetime of a FDB future is tied to the lifetime of the transaction created within db.run and db.Transact, I was wondering if

  1. At the C API level, are watch futures special in some way that allows it to safely stay alive after the transaction has been destroyed? (or)

  2. Is it just the case that Java and Go binding tester just relies on GC not to destroy the transaction object till the future is alive?

2

The lifetime of a native future in Java is not exactly tied to the transaction, but rather the completion of that future (I don’t recall for Go, but I suspect this is true there too). In most cases an incomplete future is automatically failed when the transaction that created it is destroyed, so that does have the effect of freeing the native memory. This is not true for the watch futures here, though, which are not failed when the transaction completes or is destroyed.

3

It’s also probably worth noting here that it is fundamental to the semantics of a watch that the future be able to outlive the transaction it was created in. In particular, the idea behind a watch is that the watcher get notified when the watched key is changed, including some time in the future after the original transaction is long gone.

4

Thanks @ajbeamon @alloc for the replies. :slight_smile:

I’ve a related question regarding correct use of watches and retry loop.

Lets say I’ve a watch future w0. Now if I’ve a code with the following structure, where in a subsequent transaction, I do

db.run(|tr| {
  let _ = w0.join()?;

  // an error occurs that causes retry
});

In the above code, w0.join() blocks the thread till the watch future resolves. Once w0 future resolves, before proceeding, it also takes ownership of the future and drops it (by calling fdb_future_destroy).

Now if some action happens later in the function closure body, that causes a retry, db.run will retry the transaction thereby causing a join() on a possibly destroyed future.

For normal futures this behavior is okay because when using safe Rust, futures can’t leak out of the function closure. They are all destroyed before retry occurs.

So, the question is,

  1. Is it safe to call fdb_future_block_until_ready on a watch FDB future (w0) that has already been destroyed? (I suspect not, but I am asking to confirm)

  2. Suppose I don’t destroy w0 after the first fdb_future_block_until_ready resolves, it it safe to call fdb_future_block_until_ready on w0 multiple times?

  3. In the above pattern, it it my responsibility to ensure that I don’t destroy w0 in function closure, but do it outside of db.run(...)?

5

  1. No.

  2. Calling fdb_future_block_until_ready multiple times is safe.

  3. I would think that the correct pattern would be to do:

    let _ = w0.join()?;
db.run(|tr| {
  // an error occurs that causes retry
});

    As waiting for the watch to fire (indicating that the watched value has changed) doesn’t have anything to do with the transactional body that you’re about to execute in response to being informed that the watched value has changed.

6

Thanks @alexmiller for the reply. :slight_smile:

Wouldn’t we still need to put w0 inside a retry loop, like so?

db.run(|tr| {
   w0.join()?
});

// Verify that `w0` correctly resolved without any errors.

db.run(|tr| {
  // an error that causes retry
});
7

No. There’s nothing to retry.

The watch is created as part of a transaction, so that you can create a watch transactionally. (ie. you can decide if a watch is needed based on a correct and up-to-date understanding of the database state.)

After that point, there’s no need for a transaction. It’s just a future that completes when the value has changed in the database.

8

Thanks @alexmiller.

I see.

I was wondering if there was some reason why an error from watch future was being sent to on_error in a newly created transaction in Go and Java stack tester, rather than just being ignored (ignored in the sense that bubbling the error up to the application)?

Basically I want to understand the right way of handling any errors that occurs when w0.join() returns an error.

// Go
e := watch.Get()
if e != nil {
	switch e := e.(type) {
	case fdb.Error:
		tr, tr_error := db.CreateTransaction()
		if tr_error != nil {
			panic(tr_error)
		}
		tr.OnError(e).MustGet()
	default:
		panic(e)
	}
}

// Java
try {
	w.join();
	if(!expected) {
		throw new IllegalStateException("A watch triggered too early");
	}
}
catch(FDBException e) {
	Transaction tr = db.createTransaction();
	tr.onError(e).join();
	return false;
}