Key-val sizes in the Record Layer

The Record Layer paper says the following:

To avoid exposing FoundationDB’s limits on key and value sizes to clients, the Record Layer splits large records across a set of contiguous keys and splices them back together when deserializing split records. A special type of split, immediately preceding each record, holds the commit version of the record’s last modification; it is returned with the record on every read.

Seems like a developer doen’t have to worry about key-val sizes with Record Layer, or is there any catch here?

Does it mean that the total data size in a transaction (<= 10 MB) is also not an issue here?

Thanks

Hm, the feature alluded to in the paper there is just that singe records are split across multiple key-value pairs, so single records are not subject to the 100 kB “max value” size limit.

The “catch” is:

  1. The Record Layer will choose to split your record so that each value (except the last) is 100 kB in size (right up to the limit). It’s possible that the key value store would do better if the data were split across more, smaller keys; it’s also possible that 100 kB is optimal, but we haven’t actually done the work to test that.
  2. Records must be updated in a single transaction, and the work done updating the record is counted against your transaction usage. As a result, you probably can’t write a record larger than 10 MB. The Record Layer won’t enforce that explicitly, though, so you won’t get a “record too big” error, but you will get a “transaction too large” error.
  3. On more of a data modeling level, Record Layer reads and writes are done at the record level, and (with the exception of covering indexes, where the “required fields” can be satisfied by looking at index data), so if one writes large records, any time one needs to read the record, the Record Layer will load the full record from the cluster, and any time one needs to update it, the Record Layer will save the full record again. If you have relatively cold data, that may be fine, but if parts of your record are updated or accessed more frequently than others, it may make sense to split that out into its own record.

I alluded to this above, but just to be explicit, transactions in the Record Layer are still subject to the 10 MB limit (and probably should be limited to less than 1 MB in size).

Thanks @alloc, this helps :slight_smile:

Hey @alloc,

just had a few questions regarding value size and transaction size in Record Layer.

Since there is no value size limit on a particular record and a record greater than 100kB is split into multiple key-value pairs,with each value but the last being 100kB in size and last less than 100kB, So can this size value of “100kB” be controlled by any means for example say VALUE_SIZE_LIMIT knob on the native FDB side.

Since Record Layer itself does not explicitly checks for Transaction exceeding the max size (default 10MB), So it means if we set the TRANSACTION_SIZE_LIMIT knob on the FDB native side then Record Layer should allow to persist a transaction with say a record exceeding 10MB in size. I know transactions with size greater than 10MB may degrade the performance a bit and we are fine with it as they are very infrequent but I am assuming here that the correctness won’t be compromised right?

Also for FoundationDB Java Client for Record Layer, I was not able to find an API that could help me to set the modified NetworkOptions with changed aforementioned knob’s values. Can anyone help me if I am missing something here?

Thanks

So can this size value of “100kB” be controlled by any means for example say VALUE_SIZE_LIMIT knob on the native FDB side.

Not today. The chunking size is currently defined here https://github.com/FoundationDB/fdb-record-layer/blob/1b260d4b64f42b44363a8ff7b5ce281702ca6920/fdb-record-layer-core/src/main/java/com/apple/foundationdb/record/provider/foundationdb/SplitHelper.java#L74. It should be possible to plumb through a configuration for this (possibly a setting in the record store itself), but I’m not sure I understand the value in reducing this limit, as reducing it means potentially more key/value pairs which would ultimately correspond to more overall storage space.

So it means if we set the TRANSACTION_SIZE_LIMIT knob on the FDB native side then Record Layer should allow to persist a transaction with say a record exceeding 10MB in size

Maybe someone else more familiar with the server side can chime in, but I am unaware of any knob in the server, much less the client, that would raise the transaction size limit (see Known Limitations — FoundationDB 7.1).

I know transactions with size greater than 10MB may degrade the performance a bit

Even 10MB transactions can be a pretty heavy burden on FDB under high load scenarios. If you have any concerns about high concurrency/high throughput workloads, you may overall be better off with more smaller transactions.