If cluster A is periodically taking snapshots, at some point in time we may want to restore one of these snapshots to cluster B. By the design of the snapshot feature, the restored cluster’s keyspace looks exactly like that of the original cluster. Unlike with traditional backups, even the system keyspace is identical on cluster B. This means we cannot use any system keys to differentiate between a cluster being restored from a snapshot and a cluster that is going through recovery while being snapshotted. There are several reasons we’d like to distinguish between these cases:
- The system keyspace used by backup agents is no longer relevant to cluster B, and should be cleared before backup agents start reading this keyspace (Issue 3873)
- We need to do additional setup (i.e. applying a incremental backup) to cluster B before it is ready to handle client workload. Ideally we could lock cluster B in the initial recovery transaction, but we also need to avoid locking cluster A if it happens to go through a recovery while snapshotting.
To get around the above two issues, our current approach is to solve these problems at the operational level, instead of natively in FoundationDB. However, we were wondering if anyone has any other ideas for handling this differentiation natively in FoundationDB? Thank you.