C api add_conflict_range seems not work

Development
1

fdb_transaction_add_conflict_range not work as expected?

here is my unit test

        FDBDatabase* db;  // Simplify the code
        // step 1: set key1 value1
        {
            // create transaction tr
            fdb_transaction_set(tr, (uint8_t*)"key1", 4, (uint8_t*)"value1", 6);
           //  commit tr
        }
        // step 2: get read version
        int64_t version;
        {
            FDBTransaction* tr = nullptr;
            fdb_error_t err = fdb_database_create_transaction(db, &tr);
            ASSERT_EQ(err, 0);
            FDBFuture* future = fdb_transaction_get_read_version(tr);
            err = fdb_future_block_until_ready(future);
            ASSERT_EQ(err, 0);
            err = fdb_future_get_int64(future, &version);
            ASSERT_EQ(err, 0);
            fdb_future_destroy(future);
            fdb_transaction_destroy(tr);
        }
        // step 3: rewrite key1 ----- set key1 value2
        {
            // create transaction tr
            fdb_transaction_set(tr, (uint8_t*)"key1", 4, (uint8_t*)"value2", 6);
           //  commit tr
        }
        // step 4: expect a write conflict with step 3 when use step 2's read version
        FDBTransaction* txn1_tr = nullptr;
        {
            fdb_error_t err = fdb_database_create_transaction(db, &txn1_tr);
            ASSERT_EQ(err, 0);
            // set read version from step 2
            fdb_transaction_set_read_version(txn1_tr, version);
            fdb_transaction_set(txn1_tr, (uint8_t*)"key1", 4, (uint8_t*)"value3", 6);
            ASSERT_EQ(err, 0);
            ASSERT_EQ(fdb_transaction_add_conflict_range(txn1_tr, (uint8_t*)"key0", 4, (uint8_t*)"key2", 4, FDB_CONFLICT_RANGE_TYPE_READ), 0);
            // commit txn1
            err = WaitError(fdb_transaction_commit(txn1_tr));
            ASSERT_EQ(err, 1020 /*transaction not committed due to conflict with another transaction*/);
        }

In Document, FDB_CONFLICT_RANGE_TYPE_READ means read the range, step 4 should be conflicted with step 3 when use read version from step 2.

However, the ASSERT_EQ(err, 1020 /*transaction not committed due to conflict with another transaction*/); is failed, err is 0(means success).

Is there something wrong with my usage?

2

This is kind of subtle, but I believe the problem is that you’re adding the read conflict range after setting key1 in your final transaction.

The way that the conflict range logic works is that there are data structures maintained by the FDB client indicating what ranges of keys have been read and which have been written. When you add a read-conflict range it, (1) collapses the range with other pre-existing conflict ranges (so that if you have a read conflict range from [a, c) and you add one from [b, d), it collapses that to a single range from [a, d)) and (2) removes any overwritten keys or key ranges from the read conflict range as the value seen in the transaction is consistent with the uncommitted version not the version associated with the transaction’s read version. So, in this case, you’ve got a write on key1 and then read conflict ranges [key0, key1) and [key1+\x00, key2), which means that it won’t conflict with the third transaction’s write at key1.

This behavior is a bit non-intuitive, especially when the user adds explicit conflict ranges. This is somewhat more understandable when conflict ranges come from actual reads and writes.

3

I always thought fdb_transaction_add_conflict_range(..., FDB_CONFLICT_RANGE_TYPE_READ) was equivalent to fdb_transaction_get_range on txn conflict detect.

I want to guarantee no one modify the key1 after I got the read_version, then I can modify key1. (just like optimistic lock)

Is there any way can achieve above requirement?

My current solution is replacing the fdb_transaction_add_conflict_range with fdb_transaction_get, and do not wait the fdb_transaction_get’s future ready, commit the txn immediately. Example code(step 4) like follow:

        // step 4: expect a write conflict with step 3 when use step 2's read version
        FDBTransaction* txn1_tr = nullptr;
        {
            fdb_error_t err = fdb_database_create_transaction(db, &txn1_tr);
            ASSERT_EQ(err, 0);
            // set read version from step 2
            fdb_transaction_set_read_version(txn1_tr, version);
            fdb_transaction_set(txn1_tr, (uint8_t*)"key1", 4, (uint8_t*)"value3", 6);
            ASSERT_EQ(err, 0);
-           ASSERT_EQ(fdb_transaction_add_conflict_range(txn1_tr, (uint8_t*)"key0", 4, (uint8_t*)"key2", 4, FDB_CONFLICT_RANGE_TYPE_READ), 0);
+           FDBFuture* future = fdb_transaction_get(txn1_tr, (uint8_t*)"key1", 4);
+           // not care about the value of "key1"
            // commit txn1
            err = WaitError(fdb_transaction_commit(txn1_tr));
            ASSERT_EQ(err, 1020 /*transaction not committed due to conflict with another transaction*/);
        }
4

My suggestion in this case would be to move the call to add_conflict_range to be before the set call. If I’m right about what is going on, this should result in a read conflict range being added that includes key1.

I’ll admit that this surprised me that this worked. I think what’s going on has to do with how the updateConflictMap methods work:

The second method is (I believe) responsible for adding the read conflict ranges if one calls the add_conflict_range method explicitly or if one does a get_range request. In this case, it iterates through the WriteMap to find overlapping gaps with the range, and it only adds read conflict ranges in the gaps. The first method is (again, I believe) the method that adds the conflict range if you read a single key via get, and in this case, its logic for whether it adds the read conflict key given an existing entry in the write map appears to be slightly different. So it could be that it’s adding the read conflict key here but only if you call get.

Now, whether it should have added the conflict range… it seems like if it’s reading the uncommitted value from the write cache, as I believe it will, then it shouldn’t add the conflict range, for the same reason that get_range and add_conflict_range don’t.

I believe that it actually is. If you want to confirm, you could try issuing a get_range from key0 to key2 instead of calling get or add_conflict_range and then wait on the result. (Unlike with a single key get, the waiting here is actually important because the read conflict ranges added can be different depending on what data actually gets read during a get_range.) I’d expect the same behavior as you observed as when you called add_conflict_range.

If anything, I actually think that departing from the behavior of get_range would make sense. That is, it seems like it would be less error prone if calling add_conflict_range always added the conflict range, even if there are mutations in the range.

5

It seems work for me. Thanks a lot for your suggestion.

This is a very reasonable explanation. Nice design. I need to think more carefully next time.

1 Like