I’m currently trying to add FoundationDB support to an open source project (https://github.com/locationtech/geowave) and am currently using the Java API bindings to try to run it. However, when I try to run FDB.selectAPIVersion(610), I get a java.lang.UnsatisfiedLinkError: and realized that the line I’m failing on is JNIUtil.loadLibrary("fdb_java");.
My guess for why this error is happening is because I currently don’t have the FoundationDB client libraries installed on my computer. Is there a way to bypass this? Ideally, users who use this project would not have to actually install the client libraries themselves.
Hm, interesting that you’d be failing on the fdb_java line. That library should be included within the jar (unless you’re running on some system we don’t support, but there is a Linux so file, a macOS dylib, and a Windows DLL included).
There is also separately an fdb_c dependency that you’ll need to include. There’s no way to run the FoundationDB client without having the dependency around (short of running a separate proxy service), but you don’t need the user to have installed the library, necessarily. On Linux, you should be able to set the LD_LIBRARY_PATH environment variable to include the path to a directory where you have included the library. Less tested, but also, I think on all platforms, if you set the FDB_LIBRARY_PATH_FDB_C Java system property to the path of the library, then I think that, too, should load the C library. (Note that this is undocumented, and I’m not sure if it was really intended to be something use in real environments.)
Where ${version} is the FDB version, ${os} is your operating system (linux, macOS, or windows), and ${ext} is the platform specific dynamic library file extension.
It would probably be convenient to create artifacts for the Java library that include the actual client library as well under a different artifact id. No real reason to make it a separate install if all you need is the .so.
I believe the reason is that you need to match the client C library version with the server C library version (and/or use the multi-version client in order to have no client downtime during an upgrade). Separating the C library from the Java artifact allows the user to then upgrade the C library like they would update some other part of their “config” information rather than needing to update their Java dependency.
That being said, keeping them separate is somewhat of a hassle and keeps the learning curve for new people relatively high, so it might make sense to have some kind of separate dependency with them included.
This might be crazy but how about a way to download the client libraries from the server. Been thinking about this for a different database recently and am probably going to implement it.
I think that could make sense as an option, though it would likely require adding a new HTTP interface and more information in the cluster file on how to discover it. Part of the reason we haven’t pursued that is because of security and reliability concerns around dynamically injecting code at run time, but if there are members of the community that want to accept that trade-off I don’t see anything unreasonable about it.
Hi! Nodejs foundationdb maintainer here. I’ve wanted this for a long time for the nodejs bindings too. (Usually nodejs libraries should just work out of the box!). This is the #1 issue for people just getting started, since any project which depends on the node foundationdb bindings in any way can’t even npm install without a working libfdb_c.
@spullara If you add a way for the server to send the client the libfdb_c library in some way, I’ll happily integrate it into the node bindings too. Another approach (rather than running an http server) would be to have an API to allow the server to tell the client the URL for an appropriate library file from the downloads page along with a checksum.
That approach works as long as the system that is using the driver can reach the internet. One nice thing about distributing from the server is that we know the client can reach it otherwise it won’t be able to connect either.
If we used foundationdb.org as a root of trust, one could theoretically arrange…
libfdb_c.so available for download on foundationdb.org as a .signature section that contains a signature verifying that it is from the official distribution
The public key for this signature is baked into fdbserver and bindings
Bindings verify that the libfdb_c that they receive is signed using the embedded public key
Would resolve the security concerns here, I think. However, that’s a ton of security/cryptography heavy code that I’m not sure I’d even be personally comfortable writing…
The crypto part of that isn’t as bad as it sounds. It just depends on signatures. I’d probably use libsodium’s crypto_generichash, which is explicitly marked as being appropriate for file integrity checking. Even just using SHA256 would probably be fine, but I’d want to check that thats safe against length extension attacks. (I trust that libsodium is definitely safe, since the authors know what they’re doing. But it is another dependancy though.)
@spullara I agree and I hear you. On the flip side, one nice thing about pointing to the website is we don’t bloat up the server with a bunch of extra binaries. To make this robust we’d need to ship copies of fdb_c for each platform foundationdb supports. At a minimum we’d want the libraries for windows, mac and linux, and mac version alone is 7mb in size.
Oh, yeah, this does sound way easier if it’s not trying to mentally contort myself to understand an openssl API. It looks like crypto_sign_* is what I wanted to use the official builders as a root of trust.
I think someone would next need to sit down and write up a document proposing exactly how clients would discover and download libfdb_c versions from servers in a way that fits into our client protocol, would scale (ie. cluster wouldn’t die if 10K clients show up at once), and have a sane operational story.
There’s also a “competing” proposal floating around to solve this problem via offering a well defined gRPC or similar interface, and a new “Client API Proxy” role that translates this API into FDB’s native protocol. This would leave binding authors with essentially two different ways of speaking to FDB, with the native library offering lower latency, and the gRPC interface offering libfdb_c freedom.
