Everything about GetMappedRange

Development FoundationDB Core
,
1

GetMappedRange is an experimental feature introduced in FDB 7.1. It is intended to improve the client throughput and reduce latency for a commonly used traffic pattern. An experiment with Record Layer shows that this optimization can get 4x client throughput on a certain workload.

Because this feature is under active development, there have been interests and questions from the community. To summarize everything I know about it, I wrote a document and published here. It describes what this feature is for, how to use it, and the known issues (or future work).

If you have any questions, please feel free to reply to this post. Itā€™s the best way to reach out to me and future developers of this feature.

6 Likes
2

Having read the document, this feature should fit my use case quite well. I have a custom layer that does indexing and it looks like I can easily rewrite my getv-using-index function to make use of GetMappedRange and it should improve performance significantly. It is a very common usage pattern in my app.

No questions, posting this just so that it doesnā€™t seem like nobody cares. We do :slight_smile:

3 Likes
3

I think Iā€™ve run into a bug when using GetMappedRange and Tenants. I discovered the issue while adding 7.1 support for Tokio/Rust bindings.

Here is the example rust code. It is very similar to your Java and C++ tests.

I am using FDB version 7.1.3 and starting the cluster using configure new single memory tenant_mode=optional_experimental option.

This code works fine when there are no tenants in the cluster.

However, when I add a tenant using createtenant in the CLI, and then run the same code, it errors with FdbError { error_code: 2130 } (tenant_name_required).

When I remove the tenant using deletetenant in the CLI, the code works as expected.

Line 240 is where the error is occurring.

while let Some(x) = mapped_range_stream.next().await {
    let (kv, mapped_range, mapped_kvs) = x?.into_parts();

    // ...
}

From a C API perspective, I think the error is occurring when FdbFutureMappedKeyValueArray returned by fdb_transaction_get_mapped_range tries to resolve for the first time.

Please let me know if you are able to replicate this in Java/C++.

4

The tenant_name_required error is thrown when reading a mapped range in the following circumstances:

  1. The tenant mode is required and you are running a raw access transaction (you set the raw access option or one of the system keys options).
  2. The tenant mode is optional and you try to read a range that includes a tenantā€™s key range with a raw access transaction (any transaction like above or one that doesnā€™t declare a tenant).

In other words, you canā€™t use mapped ranges with tenants if you are using raw transactions.

Itā€™s certainly possible this is failing in some other circumstance, and if so thatā€™s probably a bug.

5

Thanks @ajbeamon for the reply.

In this case, AFAICT the tenant mode is optional and I donā€™t think I am reading tenant key range. The key range that is read begins with ("prefix", ...).

6

Got it, thanks for the report. Looking at the logic for this tenant name required check, it seems I didnā€™t quite complete it. As far as I know weā€™re not currently evaluating any use-cases that rely on both features, so I guess this slipped under the radar.

Let me try to add a test to reproduce it and then Iā€™ll fix up the check.

7

Iā€™ve submitted a fix here:

2 Likes
8

Thanks @ajbeamon for the quick fix. :slight_smile:

9

I was wondering if there are any plans to land this fix in 7.1.x releases?

I currently have a test that fails because of this bug.

As I was updating my CI to 7.1.12 I checked to see if this fix has landed in the latest 7.1.x release. That does not seem to be the case, hence the question! :slight_smile:

10

Thanks for creating this feature, my use case would also benefit from GetMappedRange. Are there any plans to support GetMappedRange in the Go bindings :pray:, since currently I donā€™t think that itā€™s supported.

11

Hi @jonas, thanks for your interest. Iā€™m not aware of any plan to support it in the Go binding. Iā€™m no long working on FoundationDB for quite a while, so I donā€™t have knowledge about latest status.

I think Hao Fu is charging GetMappedRange feature now. I cannot tag him because I donā€™t know his account in the forum, but you can find him at hfu94 (Hao Fu) Ā· GitHub.

Maybe your question will get more publicity (and hopeful someone has the answer) if you ask it as a new post in the forum.

1 Like