# Record Layer **URL:** https://forums.foundationdb.org/c/using-layers/record-layer/12.md [Latest](https://forums.foundationdb.org/latest.md) · [Categories](https://forums.foundationdb.org/categories.md) · [Tags](https://forums.foundationdb.org/tags.md) --- ## [About the Record Layer category](https://forums.foundationdb.org/t/about-the-record-layer-category/939)
**Author:** [@davelester](https://forums.foundationdb.org/u/davelester)\ **Replies:** 0
Topics related to the FoundationDB Record Layer. --- ## [How to generate record schemas dynamically?](https://forums.foundationdb.org/t/how-to-generate-record-schemas-dynamically/4885)
**Author:** [@lorenzsimon](https://forums.foundationdb.org/u/lorenzsimon)\ **Replies:** 2\ **Last updated:** [May 2, 2025, 6:58am UTC](https://forums.foundationdb.org/t/how-to-generate-record-schemas-dynamically/4885 "2025-05-02T06:58:34Z")
Hi all, I am evaluating FDB record layer for a new project. One key feature is that users can register data schemas at runtime and then store data for those schemas. I have read the original record layer paper and it l… --- ## [Change primary key](https://forums.foundationdb.org/t/change-primary-key/4537)
**Author:** [@mping-exo](https://forums.foundationdb.org/u/mping-exo)\ **Replies:** 2\ **Last updated:** [July 1, 2024, 8:29pm UTC](https://forums.foundationdb.org/t/change-primary-key/4537 "2024-07-01T20:29:11Z")
Suppose I have the following protobuf message Object { string id = 1; string location = 2; string someattr = 3; } And recordlayer defines the PK as the record type plus field id Does the record layer support c… --- ## [Record Layer Primary Key Prefixes](https://forums.foundationdb.org/t/record-layer-primary-key-prefixes/4539)
**Author:** [@ArnieW](https://forums.foundationdb.org/u/ArnieW)\ **Replies:** 1\ **Last updated:** [July 1, 2024, 7:06pm UTC](https://forums.foundationdb.org/t/record-layer-primary-key-prefixes/4539 "2024-07-01T19:06:37Z")
I have the option set in protobuf for a primary key: calling loadRecord with a plain Tuple.from($email) is loading the record, which is unexpected (how does it know which primary key to append?) What is the proper … --- ## [Record layer: deleteRecordsWhere](https://forums.foundationdb.org/t/record-layer-deleterecordswhere/4356)
**Author:** [@dispalt](https://forums.foundationdb.org/u/dispalt)\ **Replies:** 5\ **Last updated:** [June 3, 2024, 6:43pm UTC](https://forums.foundationdb.org/t/record-layer-deleterecordswhere/4356 "2024-06-03T18:43:45Z")
I am trying to get deleteRecordsWhere (and the async variant) to work but I am running into trouble. It seems that it works pretty different than query and isn’t maybe as well covered with tests? I am not criticizing ju… --- ## [Thoughts from using and deploying the Record Layer as a learning exercise](https://forums.foundationdb.org/t/thoughts-from-using-and-deploying-the-record-layer-as-a-learning-exercise/4478)
**Author:** [@nbirkenshaw](https://forums.foundationdb.org/u/nbirkenshaw)\ **Replies:** 0\ **Last updated:** [May 14, 2024, 7:58pm UTC](https://forums.foundationdb.org/t/thoughts-from-using-and-deploying-the-record-layer-as-a-learning-exercise/4478 "2024-05-14T19:58:48Z")
Hi all, I’ve recently been through the exercise of developing and deploying some game leaderboards using the record layer, which have now seem some use, and I can reasonably say they work. This actually uses the TimeWind… --- ## [Different message types with same unqualified name](https://forums.foundationdb.org/t/different-message-types-with-same-unqualified-name/4400)
**Author:** [@stacktracer](https://forums.foundationdb.org/u/stacktracer)\ **Replies:** 1\ **Last updated:** [March 25, 2024, 4:27pm UTC](https://forums.foundationdb.org/t/different-message-types-with-same-unqualified-name/4400 "2024-03-25T16:27:17Z")
Is there any way to use Record Layer with protobuf message types that are in different protobuf packages, but have the same unqualified message name? For example: message MyUnion { option (com.apple.foundationdb.recor… --- ## [Ratekeeper limits dropping substantially with bulk writes](https://forums.foundationdb.org/t/ratekeeper-limits-dropping-substantially-with-bulk-writes/4336)
**Author:** [@patob](https://forums.foundationdb.org/u/patob)\ **Replies:** 4\ **Last updated:** [March 12, 2024, 8:04am UTC](https://forums.foundationdb.org/t/ratekeeper-limits-dropping-substantially-with-bulk-writes/4336 "2024-03-12T08:04:23Z")
Hi all! I’ve been are seeing a huge impact on Ratekeeper’s limits when the amount of write operations increases (~2x the amount of write transactions), thus affecting our write throughput and increasing latency in the s… --- ## [Go record layer / preload cache?](https://forums.foundationdb.org/t/go-record-layer-preload-cache/4376)
**Author:** [@newhook](https://forums.foundationdb.org/u/newhook)\ **Replies:** 0\ **Last updated:** [March 7, 2024, 6:06pm UTC](https://forums.foundationdb.org/t/go-record-layer-preload-cache/4376 "2024-03-07T18:06:55Z")
I’ve been working on a project for some time (which we will open source as it nears completion) to port the fdb-record-layer to go, based on experience we have with a similar closed source library that I wrote with simil… --- ## [Secondary Index on the presence of an optional message field](https://forums.foundationdb.org/t/secondary-index-on-the-presence-of-an-optional-message-field/4213)
**Author:** [@timcschmidt](https://forums.foundationdb.org/u/timcschmidt)\ **Replies:** 2\ **Last updated:** [January 5, 2024, 11:18am UTC](https://forums.foundationdb.org/t/secondary-index-on-the-presence-of-an-optional-message-field/4213 "2024-01-05T11:18:57Z")
First of all, sorry for the basic question, but this was not clear to me from the documentation. If I have a top-level message/record with optional message fields (e.g. in a oneof) message TopLevel { \ … --- ## [Use the fdb-record-layer without using protobuf](https://forums.foundationdb.org/t/use-the-fdb-record-layer-without-using-protobuf/4306)
**Author:** [@whisper-bye](https://forums.foundationdb.org/u/whisper-bye)\ **Replies:** 4\ **Last updated:** [December 22, 2023, 5:31pm UTC](https://forums.foundationdb.org/t/use-the-fdb-record-layer-without-using-protobuf/4306 "2023-12-22T17:31:59Z")
I don’t want to introduce protobuf into my project as it requires writing proto files. Is it possible to use my own serialization format, such as Kryo, with fdb-record-layer? --- ## [Querying an Enum Value](https://forums.foundationdb.org/t/querying-an-enum-value/1698)
**Author:** [@rickysaltzer](https://forums.foundationdb.org/u/rickysaltzer)\ **Replies:** 3\ **Last updated:** [July 5, 2023, 2:28pm UTC](https://forums.foundationdb.org/t/querying-an-enum-value/1698 "2023-07-05T14:28:08Z")
Hi There - I was curious if there were any examples on how to query an enum value using the QueryComponent. I’m having a bit of trouble: Query.field("aggregationType").equalsValue(MetricExtractorProto.AggregationType.N… --- ## [recordLayer implementation](https://forums.foundationdb.org/t/recordlayer-implementation/3707)
**Author:** [@laita](https://forums.foundationdb.org/u/laita)\ **Replies:** 0\ **Last updated:** [December 23, 2022, 9:13am UTC](https://forums.foundationdb.org/t/recordlayer-implementation/3707 "2022-12-23T09:13:57Z")
Hello, we use foundationdb in our project and are satisfied with scaling and resiliency, in future developments I am interested in multi tenancy options from recordlayer. I was wondering of java choice and if it is inte… --- ## [Extending the Record layer query and planner](https://forums.foundationdb.org/t/extending-the-record-layer-query-and-planner/3629)
**Author:** [@omidb](https://forums.foundationdb.org/u/omidb)\ **Replies:** 1\ **Last updated:** [October 28, 2022, 12:38pm UTC](https://forums.foundationdb.org/t/extending-the-record-layer-query-and-planner/3629 "2022-10-28T12:38:37Z")
Hi, I’ve looked into extending the Record layer for other use cases and I found fdb-record-layer/Extending.md at main · FoundationDB/fdb-record-layer · GitHub I was wondering if there are any examples of extending the … --- ## [KeySpace and FDB Directory Layer](https://forums.foundationdb.org/t/keyspace-and-fdb-directory-layer/3585)
**Author:** [@randyhu](https://forums.foundationdb.org/u/randyhu)\ **Replies:** 3\ **Last updated:** [October 12, 2022, 9:36am UTC](https://forums.foundationdb.org/t/keyspace-and-fdb-directory-layer/3585 "2022-10-12T09:36:00Z")
What’s the relationship between FoundationDB’s KeySpace and Directory Layer? Is directory in KeySpace eventually implemented with directory layer? The motivation of the question is for hybrid storage schema on same Foun… --- ## [Metric Collection for Record Layer](https://forums.foundationdb.org/t/metric-collection-for-record-layer/2052)
**Author:** [@mukul](https://forums.foundationdb.org/u/mukul)\ **Replies:** 2\ **Last updated:** [May 31, 2022, 2:58am UTC](https://forums.foundationdb.org/t/metric-collection-for-record-layer/2052 "2022-05-31T02:58:22Z")
I haven’t found any documentation for APIs which exposes the different metrics collected by FoundationDB Record Layer. The only source for metrics I have discovered so far is the FDB Status json file. So, for Record Lay… --- ## [Marking a message as \`NESTED\`](https://forums.foundationdb.org/t/marking-a-message-as-nested/3322)
**Author:** [@rajivr](https://forums.foundationdb.org/u/rajivr)\ **Replies:** 2\ **Last updated:** [May 16, 2022, 11:16pm UTC](https://forums.foundationdb.org/t/marking-a-message-as-nested/3322 "2022-05-16T23:16:54Z")
When using Record Layer, I was wondering when should a message be marked as NESTED? For example in the following definition for a Record Store, message MyInnerRecord1 { option (com.apple.foundationdb.record.record)… --- ## [Record Store, Versionstamp and Cluster Migration](https://forums.foundationdb.org/t/record-store-versionstamp-and-cluster-migration/3292)
**Author:** [@rajivr](https://forums.foundationdb.org/u/rajivr)\ **Replies:** 0\ **Last updated:** [April 30, 2022, 5:24am UTC](https://forums.foundationdb.org/t/record-store-versionstamp-and-cluster-migration/3292 "2022-04-30T05:24:13Z")
I have two questions regarding cluster migration of record stores. Is there some way in which the record stores can be put into “maintenance” mode prior to performing cluster migration? As I was reading SplitHelper … --- ## [Ignoring Presence of New Indexes](https://forums.foundationdb.org/t/ignoring-presence-of-new-indexes/3146)
**Author:** [@ricky.saltzer](https://forums.foundationdb.org/u/ricky.saltzer)\ **Replies:** 3\ **Last updated:** [February 1, 2022, 7:17pm UTC](https://forums.foundationdb.org/t/ignoring-presence-of-new-indexes/3146 "2022-02-01T19:17:25Z")
Hi there - When running multiple RecordLayer services we’ve come across an issue regarding adding newer indexes. If service A restarts and adds a new index record, then service B fails to service requests with a Record… --- ## [Schema evolution with multiple services](https://forums.foundationdb.org/t/schema-evolution-with-multiple-services/2887)
**Author:** [@panghy](https://forums.foundationdb.org/u/panghy)\ **Replies:** 4\ **Last updated:** [November 3, 2021, 2:20am UTC](https://forums.foundationdb.org/t/schema-evolution-with-multiple-services/2887 "2021-11-03T02:20:32Z")
It seems like the record layer currently expects all nodes to upgrade or downgrade at the same time, and even with the usage of FDBMetaDataStore, there is no easy way to revert an upgrade (or otherwise run compatible but… --- ## [Using Record Layer via other languages, e.g. Python](https://forums.foundationdb.org/t/using-record-layer-via-other-languages-e-g-python/2882)
**Author:** [@omidb](https://forums.foundationdb.org/u/omidb)\ **Replies:** 6\ **Last updated:** [September 9, 2021, 5:21pm UTC](https://forums.foundationdb.org/t/using-record-layer-via-other-languages-e-g-python/2882 "2021-09-09T17:21:06Z")
Hi, I was wondering how one can use the Record layer via other languages such as Python? Is the best way for it to build a communication layer such (gRPC, etc.)? what are the challenges to do something like that? --- ## [Record Layer Java 11 Dependency Upgrade](https://forums.foundationdb.org/t/record-layer-java-11-dependency-upgrade/2879)
**Author:** [@alloc](https://forums.foundationdb.org/u/alloc)\ **Replies:** 0\ **Last updated:** [September 3, 2021, 9:45pm UTC](https://forums.foundationdb.org/t/record-layer-java-11-dependency-upgrade/2879 "2021-09-03T21:45:36Z")
At the moment, the Record Layer currently has Java 8 as its current source and binary compatibility target, though it uses the Java 11 compiler to build and is already compatible with a Java 11 runtime. We are considerin… --- ## [deleteRecordAsync does not delete secondary index](https://forums.foundationdb.org/t/deleterecordasync-does-not-delete-secondary-index/2797)
**Author:** [@sergey](https://forums.foundationdb.org/u/sergey)\ **Replies:** 1\ **Last updated:** [August 5, 2021, 12:16pm UTC](https://forums.foundationdb.org/t/deleterecordasync-does-not-delete-secondary-index/2797 "2021-08-05T12:16:12Z")
Hi everyone! I see an issue using Record Layer and do not know how could I fix it properly. I scan in a given recordStore by secondary index (let’s call it expiration\_timestamp) and getting primary keys that using later… --- ## [Torture test gRPC facade w/ Record Layer](https://forums.foundationdb.org/t/torture-test-grpc-facade-w-record-layer/2831)
**Author:** [@panghy](https://forums.foundationdb.org/u/panghy)\ **Replies:** 2\ **Last updated:** [August 2, 2021, 4:58pm UTC](https://forums.foundationdb.org/t/torture-test-grpc-facade-w-record-layer/2831 "2021-08-02T16:58:46Z")
Had been trying to find a good torture test for the the Lionrock gRPC facade for FoundationDB and decided to go down the path of building a gRPC Record Layer client: https://mvnrepository.com/artifact/io.github.panghy.li… --- ## [scanRecords without deserialization](https://forums.foundationdb.org/t/scanrecords-without-deserialization/2801)
**Author:** [@arnaudgeiser](https://forums.foundationdb.org/u/arnaudgeiser)\ **Replies:** 3\ **Last updated:** [July 29, 2021, 6:26am UTC](https://forums.foundationdb.org/t/scanrecords-without-deserialization/2801 "2021-07-29T06:26:43Z")
Hello everyone, We performed benchmarks based on FDB Record Layer (FDBRecordStore#scanRecords) to compare with the FDB Java API. We saw a factor of 3 time-wise in favour of the FDB Java API based on the shape of our … --- ## [3.0.X artifacts](https://forums.foundationdb.org/t/3-0-x-artifacts/2824)
**Author:** [@ngbinh](https://forums.foundationdb.org/u/ngbinh)\ **Replies:** 0\ **Last updated:** [July 28, 2021, 1:42am UTC](https://forums.foundationdb.org/t/3-0-x-artifacts/2824 "2021-07-28T01:42:57Z")
Hi, Is there a way to get access to the latest versions of record layer? There are 3.0.X tags on Github but I cannot find them on mavenCentral Release 3.0.198.0 · FoundationDB/fdb-record-layer · GitHub Thanks --- ## [Checking the Existence of an Index Entry](https://forums.foundationdb.org/t/checking-the-existence-of-an-index-entry/2739)
**Author:** [@ricky.saltzer](https://forums.foundationdb.org/u/ricky.saltzer)\ **Replies:** 1\ **Last updated:** [June 14, 2021, 1:46pm UTC](https://forums.foundationdb.org/t/checking-the-existence-of-an-index-entry/2739 "2021-06-14T13:46:39Z")
Using RecordLayer is there an easy way to check the existence of an index entry? I see there’s loadIndexEntryRecord which will check the index and then load the record associated with it (if it exists). I was wondering i… --- ## [Can TEXT index work with KeyExpression using groupBy?](https://forums.foundationdb.org/t/can-text-index-work-with-keyexpression-using-groupby/2599)
**Author:** [@hiepsieunhan](https://forums.foundationdb.org/u/hiepsieunhan)\ **Replies:** 2\ **Last updated:** [March 5, 2021, 10:16pm UTC](https://forums.foundationdb.org/t/can-text-index-work-with-keyexpression-using-groupby/2599 "2021-03-05T22:16:23Z")
I am using Text index in my project, the model looks like: Group { "group\_id": { "space": String, "suffix": String }, "group\_info" { "group\_name": String } } The index is defined… --- ## [How Record Layer Provides APIs to Handle Large Range Scan Longer Than 5 seconds](https://forums.foundationdb.org/t/how-record-layer-provides-apis-to-handle-large-range-scan-longer-than-5-seconds/2400)
**Author:** [@jltz](https://forums.foundationdb.org/u/jltz)\ **Replies:** 6\ **Last updated:** [December 16, 2020, 5:43pm UTC](https://forums.foundationdb.org/t/how-record-layer-provides-apis-to-handle-large-range-scan-longer-than-5-seconds/2400 "2020-12-16T17:43:49Z")
Hi, Based on what I understand from the FDB Java Binding APIs, for a getRange() query that can potentially run over 5 seconds and encounter the “transaction too old” exception, I can capture this transaction exception … --- ## [Question about FDB Record Layer's SIGMOD'19 paper](https://forums.foundationdb.org/t/question-about-fdb-record-layers-sigmod19-paper/2460)
**Author:** [@zydong](https://forums.foundationdb.org/u/zydong)\ **Replies:** 4\ **Last updated:** [December 4, 2020, 12:54pm UTC](https://forums.foundationdb.org/t/question-about-fdb-record-layers-sigmod19-paper/2460 "2020-12-04T12:54:03Z")
Hi, I’m new to FDB Record layer and decide to use it for fun. After I read the exciting sigmod’19 paper. I have some questions: When building a universe index, we need to scan all the records stored in a record store … [Next page](https://forums.foundationdb.org/c/using-layers/record-layer/12.md?page=1)