I don’t want to introduce protobuf into my project as it requires writing proto files. Is it possible to use my own serialization format, such as Kryo, with fdb-record-layer?
1 Like
Probably not.
It was decided not to attempt to abstract a serialization / introspection API and the Record Layer’s own metadata is serialized using Protobuf. So using without adding the protobuf library dependency will be hard.
Relatedly, it would be difficult to have a serialization that went straight from domain objects to serialized bytes, because the expression evaluation needed for secondary index maintenance and query predicate evaluation needs the intermediate messages.
If the problem is solely having to write .proto files, it is possible to avoid that. Protobuf file descriptors, the root of the metadata, have a builder API and the Record Layer is used successfully with only DynamicMessages of such constructed message types. A little searching might well uncover a Java annotation based library for doing that part, although we don’t have any such experience.
Thank you for your reply. Another question is whether it’s possible to replace protobuf with another binary serialization method, such as flatbuffers, by modifying the source code or using existing extension points.
The record serializers, narrowly defined, are, in fact, extensible in a way that does allow for wholesale replacement via the RecordSerializer interface. This while leaving the Descriptor-based metadata and intermediate Message representation unchanged, which may not be what you had in mind. But if all you wanted to do is change the bytes in the value side of the key-value pairs.
Thank you for your clarification.