Let’s say I have a system like a database built as an FDB layer, where data is organized into some keyspace. I’d like the layer to have the ability to trigger a FoundationDB backups (full snapshots, as well as continuous backups) to S3 on certain events (either user triggered – e.g. “backup table in database” – or automatic for system/global disaster recovery.)
At the moment, the backup tools are the only way to do this via the command line, but having an API actually exposed for this would be quite handy. Would some APIs to provide this explicitly be within scope and, if done properly, acceptable upstream?
There might be a slightly deeper complication here, of the current backup framework was written with being a continuous ongoing process in mind, rather than a single snapshot of the data. (Unless you’re running on a cloud platform, then @lihtnes implemented a disk snapshot-based backup and restore, that would require some extra tooling.). @SteavedHams would probably know if I’m wrong, or how much effort it would take to turn backup into something that would just do one snapshot again.
@john_brownlee might have some additional opinions about how this would best be done, but I don’t think we’re overall opposed for having management tasks being added to the API and invokable outside fdbcli.
Actually, whether a backup is continuous (keeps going after the first restorable snapshot) or not is just an option, the -z flag, which is off (not continuous) by default.
I believe there has been some discussion of exposing the “management API” (e.g., the things that can change cluster configuration rather than cluster data) through some API other than fdbcli. This is somewhat related to: Using the multi-version client in the CLI · Issue #1809 · apple/foundationdb · GitHub I can’t find an issue about this specific issue, though, but the management API is essentially what powers the CLI (the parts that aren’t about just getting/setting keys), so exposing that as a programmatic API would get you most of the way there.
To get backups to be started programmatically, it would require a slightly different API exposed (the fdbbackup and fdbdr command line utilities) rather than the management API, but that doesn’t sound like an unreasonable thing to upstream to me.