We’re thinking using fdb for our application. One thing we want to know is that, since we have different clients (customers) using the database, is there a way to make/restore backup based to clients? Say, is it able for each client has a client id and their own backup?
Thank you!
The short answer is “yes” but it’s up to your application to actually manage running backups and restores that are restricted to data pertaining to one of your customers.
FDB does not have a user concept. From the database’s perspective all of your user data is in one key space we call “user space” which is from “” to “\xff”. It’s up to your application to sub-divide that space into different ranges and use different ranges for your different users.
You can backup the entire database in one backup job, or you can target a specific key range and you can have many backups running, each targeting set of key ranges of your choosing. Regardless of which option you choose, restore tasks can also be restricted to a set of key ranges as well, so even if you have one backup for the entire database (the recommended option) you can still perform restores on a subset of data relevant to some subset of your users.
Thank you for the clarification. So if we give each of our customer a unique prefix (using namespace/subspace like Subspace(prefix_tuple, prefix_element)), then we can backup/restore all key-value pair starting with this prefix right?
And is it equal to your way of “Record Layer” concept described in https://www.foundationdb.org/files/record-layer-paper.pdf? I found the implementation at https://github.com/FoundationDB/fdb-record-layer outside of the official repo, has it been integrated to FDB yet? We are renting customers the database so we may need the data isolation and multi tenancy this Record Layer provides.
If you give a customer a unique prefix, you can use the -k argument to fdbbackup to back up only a restricted range of the key space, or do a backup of the whole database and use fdbrestore -k to restore only a restricted subset of the keys.
The ability to backup and restore specific ranges individually is tangential to the record layer.
Then what if we want to go with the record layer thing? Does record layer provides more data isolation and multi tenancy? That’s what we are looking for.
I’m also still a little bit confused here what’s the concept of record layer and how is this implemented and different from what you already have?
The record layer makes it significantly harder for other users of the record layer API to trample on each other, and makes representing many small databases easier than running multiple FDB instances. There is no protection against one record layer user opening and using another record layer store in the same database. In the end, this is all just keys and values on FDB, and FDB has no native multi tenancy support.
So if all of your clients are using the record layer, then it’s much harder for them to issue a stray clear_range(\x00, \xff). However, only mTLS would prevent a persistent person from doing so.
The record layer provides structured schemas for your data, schema changes, a query API, secondary indexes, etc., that are all really nice features to have when using FDB. It does not provide per-user QPS quotas, or storage size quotas, and you’d need to build those in to your application at a layer above FDB.
Yes thanks.
I’m now encountering some problem with restore. I created a snapshot and the URL is something like file:///.../backup-2019-06-27-21-39-37.925847
When I tried to describe this backup, is shows,
Restorable: False
SnapshotBytes: 0
I checked the backup directory it do have data in the snapshot folder
What files are in file:///.../backup-2019-06-27-21-39-37.925847?