How to recover FDB data based on cloud disk data

Using FoundationDB Running FoundationDB
1

Using cluster file `/mnt/config-volume/cluster-file’.

The database is unavailable; type `status’ for more information.

Welcome to the fdbcli. For help, type `help’.
fdb> status

“We are currently facing an urgent issue with our FoundationDB deployment via Helm on Kubernetes, specifically we are unable to restore data. We attempted to copy the previous cloud disk data related to the FDB components into the pods hosting the log and storage components. Following the instructions at fdb-kubernetes-operator/docs/manual/operations.md at main · FoundationDB/fdb-kubernetes-operator · GitHub, we proceeded with the data recovery process, however, the data has not been restored successfully.”

Using cluster file `/mnt/config-volume/cluster-file’.

Locking coordination state. Verify that a majority of coordination server
processes are active.

10.43.45.157:4501 (reachable)
10.43.117.22:4501 (reachable)
10.43.253.1:4501 (reachable)

Unable to locate the data distributor worker.

Unable to locate the ratekeeper worker.

2

Could you add some more details on what happened and what steps you performed? When you say disk data you mean all the data that was present under /var/fdb/data? Or did you only copy a subset of data? The steps that you linked are meant for a case where you lost the majority of the coordinators but not all of your data (thought those recovery steps will probably/eventually lead to some data loss).

Have you looked at the trace events of the cluster controller to see where/and why recovery is stuck?

3

Thank you for your reply. The previous issue has been resolved, and the data has been recovered. Now, I’m encountering a new problem. We are using the Operator version 1.36.0, with FoundationDB at version 7.1.15, all deployed via Kubernetes Helm. I’m unsure how to utilize backup data and transfer it to Alibaba Cloud OSS，
Do you have any more detailed examples or related articles?

4

Your question is how to do backups with Alibaba Cloud OSS?

There are a few backup related docs/blogpost:

I don’t know enough about Alibaba Cloud OSS to give you any details on how to set it up, but I would expect that if the service provides a S3 compatible API it should work. If you are able to find out how to configure, feel free to open a PR with the required steps :slight_smile:

5

yes ali oss supports s3.
Is it usually done through backupCluster when backing up to S3?

---- Replied Message ----

From | Johannes Scheuermann via FoundationDBnotifications@foundationdb.discoursemail.com |

  • | - |
    Date | 09/20/2024 17:10 |
    To | 13362133754@163.com |
    Cc | |
    Subject | [FoundationDB] [Using FoundationDB/Running FoundationDB] How to recover FDB data based on cloud disk data |

| johscheuer Johannes Scheuermann
September 20 |

  • | - |

Your question is how to do backups with Alibaba Cloud OSS?

There are a few backup related docs/blogpost:

I don’t know enough about Alibaba Cloud OSS to give you any details on how to set it up, but I would expect that if the service provides a S3 compatible API it should work. If you are able to find out how to configure, feel free to open a PR with the required steps :slight_smile: