Hi all,
I found some higher level language bindings support namespace concepts.
But I didn’t see it in C client APIs.
Would it be possible to use namespace or any other data isolation methods in C sides?
I do found subspace.h/subspace.cpp in the foundaitondb repo(at directory /fdbclient), does anyone know how to compile and use them?
Best Regrads,
Are you asking about being able to use the directory, subspace, and tuple layers in the C bindings? You are correct that these are only implemented in higher level bindings and not in the C bindings, though I don’t recall the exact rationale for that choice. Possibly it’s because it would have made the C interface somewhat complicated and potentially less stable, though it comes at the cost of having to implement these features (and any ongoing updates to them) in each binding.
The subspace files you found in the repository are for our Flow bindings . These are bindings that support our Flow programming language (https://apple.github.io/foundationdb/flow.html), which is basically C++ with some added support for actor-based concurrency. I’m not sure how feasible it is to actually use Flow for arbitrary projects (it should be possible, but it’s not really documented at all and likely cumbersome), so using those bindings at this point would be a challenge. However, it looks like the Subspace and Tuple layer implementations in the Flow bindings may be written in pure C++, so it’s possible you could take and use them with minimal modification in a C++ program. The directory layer is implemented in Flow, so you wouldn’t be able to do the same with it.
Thank you very much.
Allow me to ask another question, is there any way of open multiple databases in a single machine?
Those are two way I can come up with when dealing with data isolation problem.
Subspaces exist to provide a utility for converting Tuples to key prefixes, which form our namespaces. However, you could build a little tooling to manage your own prefixes (for example, you could just specify a few hard-coded prefixes for where you are storing certain data, or have some utilities to generate prefixes). See https://apple.github.io/foundationdb/developer-guide.html#namespace-management for a brief mention of this idea, and https://apple.github.io/foundationdb/data-modeling.html for some general thoughts about data modeling.
As to your question about whether it’s possible to open multiple clusters on a single machine, I’m not sure whether you’re asking if the same client can connect to multiple clusters or if the same server machines can run multiple clusters. The answer to both questions is yes. If you are instead asking whether a single cluster can contain multiple databases, that’s something we don’t support.
Thank you very much for all your reply.
Sorry, I am new to foundationdb. That’s why my question sounds vague.
Just make sure I didn’t interpret your answer in a wrong way:
Same client connect to multiple clusters:
Is this when we changing the IP:PORT in the fdb.conf file. We can have multiple clusters and these clusters may or may not come from same IP.
Same server machines can run multiple clusters:
Is this the case that same server could have multiple clusters which differentiates by the IP address.
I think what I wanna ask is the last situation that if it possible to have multiple database within the same foundaitondb service in the background(this fdbserver may contains single/multiple clusters.). Or can we open multiple foundationdb services in the background and each cluster can contains their own database. In this sense, data could be isolated into different database. But to my limited attempt, the answer may be no.
To connect to multiple clusters from one client, you would open multiple connections, each time using the cluster file for the cluster you want to connect to. In the C bindings, this would be done using fdb_create_cluster. You shouldn’t need to change the contents of any of your cluster files, and you’ll want to have a separate file for each cluster.
To run multiple clusters on the same server machines, you would need to start multiple processes on each machine with each process configured to use the cluster file for the cluster it’s a part of. You’ll also need to configure a distinct data directory and port for each process (and depending on your circumstances, you may want to use different disks for performance reasons). You don’t need to use different IP addresses, though.
An individual cluster can’t run multiple databases, but I think it should be possible to configure fdbmonitor to manage multiple clusters. You would need to specify the appropriate parameters for cluster file, data directory, and port for each process as I described above in your fdbmonitor configuration (Configuration — FoundationDB 7.1). Although it apparently doesn’t say this, you can specify a separate cluster_file parameter for each process. Another alternative is to set something up to run multiple fdbmonitor instances, one for each cluster. In some ways this would probably be easier to manage, though you’d have to provide a mechanism for those fdbmonitor instances to get run.
All of this said, running multiple databases may not be the best approach if all you are trying to do is provide two separate namespaces for your clients. It’s going to be operationally much more effort, and also our transactions would be limited to operations on a single database. If you need to transactionally make updates to both namespaces, you’d have to manage that yourself. It would likely be much easier to manage namespaces within a single database (and to be honest, even if you do split your data into two databases, I would still recommend using some kind of namespacing scheme to avoid potential issues in the future). For example, you could do something very simple and prefix your keys with something like “/namespace1/” and “/namespace2/”. This is essentially what the subspace layer is doing, except that it supports multiple types and has some nice properties with respect to ordering.
Thank you very very much for all your help.
That actually really gives me a hand in my situation.
Thanks again and best regards to you.