Fdbbackup files smaller than the actual storage size of the original database

Using FoundationDB
1

through command status details in origin database:
Data:
Replication health - Healthy
Moving data - 0.000 GB
Sum of key-value sizes - 20 MB

but the size of the kvranges in the backup file generated by the fdbbackup command is noticeably smaller than the actual storage size of the original database (the result obtained from the status details command). However, there were no errors reported during the execution of the backup process.

fdbbackup start -C /mnt/config-volume/cluster-file -d ‘file:///home/backup’ --log
The backup on tag `default’ was successfully submitted.

then
fdbbackup status -C /mnt/config-volume/cluster-file:
The previous backup on tag `default’ at file:///home/backup/backup-2024-09-30-03-44-54.440029 completed at version 6292909345356.
BackupUID: 906a2085c6b1bdfab79cd4f18f0fbbe5
BackupURL: file:///home/backup/backup-2024-09-30-03-44-54.440029

In the directory of the file named “backup-2024-09-30-03-44-54.440029”, there are three subdirectories:
kvranges logs snapshots
the kvrange size is only 1.3M which is noticeably smaller than 20M
[root@fdb-backup-agent-6cb5b6bdc7-p8xnx kvranges]# ls
snapshot.000006292889345357
[root@fdb-backup-agent-6cb5b6bdc7-p8xnx kvranges]# cd snapshot.000006292889345357/
[root@fdb-backup-agent-6cb5b6bdc7-p8xnx snapshot.000006292889345357]# ls
0
[root@fdb-backup-agent-6cb5b6bdc7-p8xnx snapshot.000006292889345357]# cd 0/
[root@fdb-backup-agent-6cb5b6bdc7-p8xnx 0]# ls
range,6292889377357,492f54d441e06f9144136b723ee28824,1048576
[root@fdb-backup-agent-6cb5b6bdc7-p8xnx 0]# ls -lh
total 1.3M
-rw-r–r–. 1 root root 1.3M Sep 30 03:44 range,6292889377357,492f54d441e06f9144136b723ee28824,1048576

2

I have very little knowledge about FoundationDB, I just want to back up all the data at the current time

3

foundationdb version is 7.1.15 and backup_agent pod version is same

4

Since your DB is small, you can dump all the key value pairs and calculate their sizes, e.g., fdbcli --exec getrange "" \xff 10000000000. The third parameter is the number of keys. Because of 5s limit, you may not read all keys at once and you may need to resume from the last key read. Most likely the total size is about 1.3M. Then you can check keys in the system keyspace, i.e., \xff to \xff\xff and its size.

Another thing to check is if you have multiple backup agents running, each one will write backup files. So the total should be counting all output files.

5

thanks for your reply, but status details command shows
Data:
Replication health - Healthy
Moving data - 0.000 GB
Sum of key-value sizes - 22 MB
Disk space used - 526 MB

6

These are the last few lines of the log file from executing the fdbbackup command, where a usage rate reaches one hundred percent, and I’m not sure what it means.

<Event Severity="10" Time="1728358698.677631" DateTime="2024-10-08T03:38:18Z" Type="FastAllocMemoryUsage" ID="0000000000000000" TotalMemory="917472" UnusedMemory="0" Utilization="100.000000%" ThreadID="2510434358476276060" Machine="172.22.25.33:266" LogGroup="default" ClientDescription="primary-7.1.15-2510434358476276060" />
<Event Severity="10" Time="1728358698.677631" DateTime="2024-10-08T03:38:18Z" Type="NetworkMetrics" ID="0000000000000000" Elapsed="0.51311" CantSleep="0" WontSleep="2" Yields="1" YieldCalls="17" YieldCallsTrue="0" RunLoopProfilingSignals="1" YieldBigStack="0" RunLoopIterations="54" TimersExecuted="62" TasksExecuted="89" ASIOEventsProcessed="84" ReadCalls="39" WriteCalls="16" ReadProbes="22" WriteProbes="0" PacketsRead="19" PacketsGenerated="20" WouldBlock="22" LaunchTime="8.10623e-06" ReactTime="7.08103e-05" DCID="[not set]" ZoneID="[not set]" MachineID="[not set]" SlowTask1M="1" PriorityBusy0="0.507966" PriorityStarvedBelow1="0.00290251" PriorityMaxStarvedBelow1="0.000918865" PriorityStarvedBelow3500="0.00290251" PriorityMaxStarvedBelow3500="0.000918865" PriorityStarvedBelow7000="0.00266075" PriorityMaxStarvedBelow7000="0.000918865" PriorityStarvedBelow7500="0.00221038" PriorityMaxStarvedBelow7500="0.000838995" PriorityStarvedBelow8500="0.00152445" PriorityMaxStarvedBelow8500="0.000838995" PriorityStarvedBelow8900="0.000720739" PriorityMaxStarvedBelow8900="8.17776e-05" PriorityStarvedBelow10500="0.000125647" PriorityMaxStarvedBelow10500="6.8903e-05" ThreadID="2510434358476276060" Machine="172.22.25.33:266" LogGroup="default" ClientDescription="primary-7.1.15-2510434358476276060" TrackLatestType="Original" />
<Event Severity="10" Time="1728358698.677631" DateTime="2024-10-08T03:38:18Z" Type="MachineMetrics" ID="0000000000000000" Elapsed="0.51311" MbpsSent="2.72627" MbpsReceived="17.0646" OutSegs="309" RetransSegs="0" CPUSeconds="0.0254015" TotalMemory="8188428288" CommittedMemory="2482495488" AvailableMemory="5705932800" DCID="[not set]" ZoneID="[not set]" MachineID="[not set]" ThreadID="2510434358476276060" Machine="172.22.25.33:266" LogGroup="default" ClientDescription="primary-7.1.15-2510434358476276060" TrackLatestType="Original" />
7

Sum of key-value sizes is only an estimate and it includes the FDB internal system keyspace (keys starting with \xff) which is not written to the backup.