We recently upgraded to FDB5.2 from FDB3 on several clusters. Only one exhibited this behavior. FDB was restarted after the new binaries and libraries were dropped in place and after some amount of time began healing except with more than double the amount of data stored:
Moving data - 32404.500 GB
Sum of key-value sizes - 8.361 TB
Disk space used - 23.588 TB
I would like to know how is 32404.500 calculated? In fact it was originally around 40T when healing began. There are 4 nodes running in double redundancy mode using the ssd-1 storage engine with 32 processes spread across them.
Is this related to the bug:
Sometimes data distribution calculated the size of a shard incorrectly. [6.0.15] https://github.com/apple/foundationdb/pull/892 by any chance?
Any help much appreciated.