In short, no. I’ll just explain how it works.
All files written to a backup are immutable. This is because the backup folder layout is the same on a local filesystem or on an object storage service using the S3 API and the latter does not allow file appends.
There is no “conversion” of anything, at any time, during the backup process. Only points at which more immutable files are written to the backup. Those files are later deleted during expiration of backup data before a point in time or during deletion of an entire backup.
Mutation logs, in version order, are continuously written to files with the logs/ folder structure such that the folder name and filename describe the version range that the log file covers. There are about 5,000 of these written per day, and their sizes are determined by the write load of the cluster.
During a Snapshot, many small chunks of Key-Value pairs are written to the kvranges/ folder at a rate targeting the Snapshot Window as a total completion time. Each file represents a sorted, contiguous set of key-value pairs read at a consistent version. The folder name indicates which Snapshot the file is a part of, and the filename indicates what version its contents were read at.
At the end of a Snapshot, a list of mutually exclusive files covering the backup’s target key range(s) is written in JSON format to a new file in the snapshots/ directory. This snapshot “manifest” file exists because the list is not simply all of the files in the kvranges/ folder under the snapshot’s subfolder *.
To perform a restore, you need
- A Snapshot file, which describes a Snapshot having a start version and an end version which are approximately Snapshot Window apart in time and having an end version less than or equal to your restore target version.
- All of the kvrange files that are listed in the Snapshot file
- All of the log files from the start version of the Snapshot file through your restore target version
So going back to the example, if your Snapshot Window is 10 days, starting at day 0, and you want to restore to a point at day 19, and your database size is holding steady at X but you modify/replace X amount of data each day, then your Snapshot file will contain a list of kvrange files describing approximately X of data and the 19 days of mutation logs you will need to restore will each contain X amount of data (the amount modified per day) and so your restore process will have to read 20 * X data.
(*) Due to the nature of working with two separate storage systems - the FDB cluster which contains backup task execution state and the backup medium where files are written to - it is fairly common for errors to cause a backup task, responsible for writing one file to logs/ or one file to kvranges/, to succeed in writing a file to the backup medium but then fail to mark the task as completed in the database. In the case of kvrange/ files this can result in files in the backup which have overlapping key ranges. This is not valid input for the restore process, so the backup process keeps a list in the database of a mutually exclusive kvrange file set and then flushes it to a JSON file in snapshots/ at the end of the snapshot.
