Developing a Read-Only FDB driver

(Devon) #1

My team would like me to develop a way to run FDB in a read-only mode, to protect against accidental writes categorically. Our goal is to be able to flip a switch and prevent writes on any specific day, without changing our code to prevent writes in general, since we sometimes still need them.

The simplest way I can think of doing this to satisfy our needs is to replace the FDBTransaction class in the java api (since we only use java) with one that throws an exception any time one of the write-specific methods from the Transaction interface is called, as long as the database has been acquired in a read-only mode. If it’s been acquired in write mode, it executes the methods normally.

But if possible we’d really like to get this upstream in some form or another so that it doesn’t have the potential to break after every release. So if there’s a preferred way to do it in the c++ code (or someone who’s already working on this) I’d be more than happy to take that approach instead. Has there been any discussion about this before?

(Alex Miller) #2

I was thinking that this could maybe be done by introducing an ALLOW_WRITES TransactionOption, and then use the support from Support setting Transaction options at the Database level to make ALLOW_WRITES=false the default after opening the database, thus making all your transactions created read-only by default.

However, it doesn’t appear that #1323 applied to all transactions options, only some? @alloc, was that intentional? Would extending it for this be fine?

(Alec Grieser) #3

Yeah, #1323 was intentionally applied to only a subset of transaction options. In particular, the only options that I added were the ones I could make a good case for as something that made sense to apply to all transactions (like retry limit). So, for example, we’d probably never want to add the set_next_write_no_conflict_range option globally.

But I think it would be reasonable to add more. The change (at an implementation level) was to add more fields to the DatabaseContext struct and to use those values to initialize the transaction options, so adding more fields could make sense. The two that were the most questionably excluded were set_disable_use_during_commit_protection and set_read_lock_aware. The former in theory might make sense to disable globally and, in fact, is disabled globally already in Java as it’s called by Database::createTransaction. Pushing that down might also make sense, though whether it’s safe kind of depends on the language bindings.

The set_read_lock_aware option, in theory, maybe should be set globally so that if one has a locked cluster, one can set it up so that one can easily read stale data from it without having to set the option manually each time. (Though maybe this is less important in FDB 6 than FDB 5 because of multi-region configurations don’t require the option be set to read from secondary regions.) The problem is that set_read_lock_aware is implemented by (1) setting lock_aware and (2) disabling commits. And one can’t “undisable” commits, so you get a kind of strange database object back.

Which I guess brings me back to the original question. The only method that must actually throw an error to ensure the transaction is read-only is commit—all other write-related operations will just be buffered locally. (Except maybe some keys in the \xff\xff keyspace? Maybe?) And the native client actually already has a way to disable commits (it’s used by set_read_lock_aware) though it appears not to be settable by the user.

Unlike the proposed option, I don’t think it can be undone, so using it as a way of providing a “read-only be default” database might not work. But your client could implement something like “on transaction creation, if I am in a read only context, disable commits”, and the only awkward thing is that doesn’t play super well with how our retry loops work.

(Devon) #4

Thanks! It sounds like the biggest issue you see is the inability to turn read-only mode off once commits have been disabled. But it should be possible to address that by defaulting the database to a write-enabled mode, and then setting the disable writes option at the database level on a per client basis, correct? I’m out of the office next week, but I’ll take a look at how set_read_lock_aware actually disables commits when I’m back.

(Alec Grieser) #5

Right. To be clear, I was really just commenting on what would happen on the client. (None of this affects the server. AFAIK, there isn’t any way of locking the database in a read only mode at the server level.) In particular, as implemented, as there’s no way for a given transaction to start allowing writes, a client who wishes to have a mix of read-only and read-write transactions from the same process can only do so by defaulting all transactions to read-write and then disabling writes (in a somewhat roundabout way) on transactions they want to be read only. It seemed borderline to me to have an option that would (surprise!) stop the client from making any commits at all, but I guess I can see the benefit of a whole process which could only read from the DB. (I don’t have too many principled objections, but it seemed like the conservative thing to do when introducing a new API would be to leave out the things that were borderline if they could be added later.) I do think that the option should be explicit (e.g., a “set read only” option on the database object) rather than implicit (e.g., turns out, setting “read lock aware” also disables commits).