# Versionstamps and DR streaming **URL:** \ **Category:** Development\ **Created:** [July 12, 2024, 2:43pm UTC](https://forums.foundationdb.org/t/versionstamps-and-dr-streaming/4564 "2024-07-12T14:43:48Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![arohner](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/arohner/32/1475_2.png) [@arohner](https://forums.foundationdb.org/u/arohner)\ **Post date:** [July 12, 2024, 2:43pm UTC](https://forums.foundationdb.org/t/versionstamps-and-dr-streaming/4564/1 "2024-07-12T14:43:48Z")
When using a DR cluster, my understanding is that the DR cluster’s keys and values will not be perfectly identical, if one uses set\_versionstamped\_key, because the DR cluster will call set\_versionstamped\_key itself, and its clock is not synchronized. Is that right? In application code, if I call `set_versionstamped_key` `get_range` (to find the key) `clear(complete_versionstamp)`, when those operations are copied over to the DR cluster, the `clear` will not be effective, because the key on disk has a different complete versionstamp. Is that correct? If so, what are my options? ---
**Author:** ![jzhou](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/jzhou/32/445_2.png) [@jzhou](https://forums.foundationdb.org/u/jzhou)\ **Post date:** [July 12, 2024, 10:44pm UTC](https://forums.foundationdb.org/t/versionstamps-and-dr-streaming/4564/2 "2024-07-12T22:44:14Z")
No, set\_versionstamped\_key won’t cause difference in the DR cluster. The reason is that versionstamp operations are converted into SET mutation at the commit proxy via `transformVersionstampMutation()` calls. DR will copy the converted mutations to the destination cluster. Thus, the destination side applies the same mutation as the source cluster. I.e., the source and the destination cluster are identical. ---
**Author:** ![miridius](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/miridius/32/1500_2.png) [@miridius](https://forums.foundationdb.org/u/miridius)\ **Post date:** [July 15, 2024, 9:34am UTC](https://forums.foundationdb.org/t/versionstamps-and-dr-streaming/4564/3 "2024-07-15T09:34:00Z")
Thanks for the answer! Are there guarantees that versionstamps continue to be unique and absolutely ordered (i.e. no new versionstamps will overlap or be older than any that previously got copied over via DR) in various recovery scenarios: 1. If we failover and make the DR cluster active? 2. If we restore the main cluster from the DR cluster by a second `fdbdr switch` or similar? 3. If we create a new cluster from an S3 backup? ---
**Author:** ![jzhou](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/jzhou/32/445_2.png) [@jzhou](https://forums.foundationdb.org/u/jzhou)\ **Post date:** [July 17, 2024, 4:37am UTC](https://forums.foundationdb.org/t/versionstamps-and-dr-streaming/4564/4 "2024-07-17T04:37:46Z")
The answer to 1 and 2 is Yes, and no guarantee for 3. 1. if `fdbdr switch` is issued, the destination cluster’s commit version will be bumped ([foundationdb/fdbclient/DatabaseBackupAgent.actor.cpp at e89cf52f6c9a96985107492bb06029495f8f9809 · apple/foundationdb · GitHub](https://github.com/apple/foundationdb/blob/e89cf52f6c9a96985107492bb06029495f8f9809/fdbclient/DatabaseBackupAgent.actor.cpp#L2784)) to be larger than the source cluster, before the switchover is done. So this will guarantee versionstamps to be monotonically increasing and unique. 2. A second `fdbdr switch` will follow the same process and a version bump. So the main cluster will have a higher version, thus providing the guarantee. 3. There is no guarantee for 3, because no such version bump exists for restore. Someone has to use `fdbcli> advanceversion` command to advance the commit version. ---
**Author:** ![danm](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/danm/32/1393_2.png) [@danm](https://forums.foundationdb.org/u/danm)\ **Post date:** [July 17, 2024, 2:17pm UTC](https://forums.foundationdb.org/t/versionstamps-and-dr-streaming/4564/5 "2024-07-17T14:17:38Z")
I notice you specifically call out `fdbdr switch`, but to our understanding in a ‘true’ disaster scenario (where the source cluster has become unavailable for some reason) you’d want to run `fdbdr abort --dstonly`. Would the same guarantees apply in that case as with `fdbdr switch`? From my reading of the code you linked it _looks_ like yes, but I’ve never written C++ before and it’s about 20 years since I last wrote C, so I don’t trust my own evaluation 😉 ---
**Author:** ![jzhou](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/jzhou/32/445_2.png) [@jzhou](https://forums.foundationdb.org/u/jzhou)\ **Post date:** [July 17, 2024, 5:33pm UTC](https://forums.foundationdb.org/t/versionstamps-and-dr-streaming/4564/6 "2024-07-17T17:33:21Z")
In case of “true” disaster, the destination cluster won’t be able to cleanly get _all_ the data from the source cluster. I.e., there could be a few seconds’ mutation data missed (if this is a concern, please consider the multi-region configuration, a different topic). When aborting the DR, the destination also does a version bump ([foundationdb/fdbclient/DatabaseBackupAgent.actor.cpp at 6cd5ad2ffe1d246e695bb51d0184e74d9bac7217 · jzhou77/foundationdb · GitHub](https://github.com/jzhou77/foundationdb/blob/6cd5ad2ffe1d246e695bb51d0184e74d9bac7217/fdbclient/DatabaseBackupAgent.actor.cpp#L2949)), and the comment says: > The +1 is because we want to make sure that a versionstamped operation can’t reuse > // the same version as an already-applied transaction. So there is a “weak” guarantee here, because the missed mutations could have other versionstamp operations.