Using Record Layer via other languages, e.g. Python

Using Layers Record Layer
1

Hi,

I was wondering how one can use the Record layer via other languages such as Python?

Is the best way for it to build a communication layer such (gRPC, etc.)? what are the challenges to do something like that?

2

There’s no native support for interacting with the Record Layer (directly) from a non-Java language. In theory, you could try to use something like, say, Jython which claims to have some kind of Java interoperability, though I wouldn’t necessarily recommend it. My guess, having not tried it, is that for languages with very strong Java interoperability (e.g., Scala) it would work fairly well; for languages that don’t, it might be a little rocky.

I suppose you could do something like try and build a gRPC (or other RPC framework) front-end to the record layer, and I think that could be successful. I think the choices you’d have would be to either:

  1. Attempt to make a general purpose database front-end that is backed by the Record Layer. (I’d also classify attempting to create a front-end that exposes an API that looks like the Record Layer API in this category.) If you do this, you have to solve problems like figuring out how a user specifies the RecordMetaData they want to use, and you have to also be sure that your API supports multi-request transactions (possibly, say, through gRPC streams) where each request needs to hit the same API server due to how FDB transactions are implemented.
  2. Expose an API that is more application specific, i.e., write a (hopefully thin) Java wrapper around your data model and then expose the operations you care about through a service API that you can then interact with with a language of your choice. I think this is ultimately less code, though its less transferable to other applications.

Both of these have some common challenges, e.g., maintaining a Java code base that is at least a little bit separate from the rest of your application and then all of the sundry problems that one gets from running a service (e.g., observability and monitoring). In both of these, you also have to decide whether your deployment model is going to be as a sidecar (e.g., each instance of your application being a Java API server with localhost-only connections and an instance of your application in your language of choice) or as a separate Java service (that your application in your language of choice connects to like it would any other service).

3

Something that I am hoping to attempt (not sure how successful I’ll be), after my current Rust bindings work, is to create a interoperable RecordLayer implementation in Rust (or atleast a subset of the APIs)

I think having multiple RecordLayer implementations would really help in the adoption of FDB.

4

You might be interested in embly/vinyl which was a previous attempt at wrapping the record layer in gRPC to make it usable from rust.

1 Like
5

You might also be interested in my attempt to wrap the record layer over grpc called Record-Store.

6

Thanks for the suggestions. Beside the challenges with multi-request transactions, I can see a world that one can expose RL API through gRPC. I really believe this can be a good addition to the RecordLayer. Is that something on the roadmap?

7

We’re using RecordLayer with Kotlin, and it works fantastic. We have a gRPC-based CRUD layer which works great since everything in RecordLayer is already Protobuf.

1 Like