# Using FoundationDB **URL:** https://forums.foundationdb.org/c/using-foundationdb/7.md?page=3 [Latest](https://forums.foundationdb.org/latest.md) · [Categories](https://forums.foundationdb.org/categories.md) · [Tags](https://forums.foundationdb.org/tags.md) **Page:** 4 --- ## [How do I debug 'non-restorable' backups?](https://forums.foundationdb.org/t/how-do-i-debug-non-restorable-backups/4940)
**Author:** [@danm](https://forums.foundationdb.org/u/danm)\ **Replies:** 0\ **Last updated:** [June 12, 2025, 8:47am UTC](https://forums.foundationdb.org/t/how-do-i-debug-non-restorable-backups/4940 "2025-06-12T08:47:53Z")
In my past experience, I’ve only ever seen a backup be Restorable: false either: before the initial snapshot is complete, or when attempting to describe it without the --deep flag (sometimes). Earlier this week we tri… --- ## [How do I run the multi-version client on MacOS arm64?](https://forums.foundationdb.org/t/how-do-i-run-the-multi-version-client-on-macos-arm64/4614)
**Author:** [@danm](https://forums.foundationdb.org/u/danm)\ **Replies:** 2\ **Last updated:** [June 12, 2025, 8:26am UTC](https://forums.foundationdb.org/t/how-do-i-run-the-multi-version-client-on-macos-arm64/4614 "2025-06-12T08:26:22Z")
We’re trying to deploy the multi-version client in our Clojure app so we can upgrade our FDB 7.1 cluster to 7.3 with minimal downtime. Because the app is Clojure, and thus runs on the JVM and has full JVM interop, we’re … --- ## [ERROR: Failed to load a required FDB API function. (2204)](https://forums.foundationdb.org/t/error-failed-to-load-a-required-fdb-api-function-2204/4404)
**Author:** [@erwan](https://forums.foundationdb.org/u/erwan)\ **Replies:** 3\ **Last updated:** [June 12, 2025, 3:41am UTC](https://forums.foundationdb.org/t/error-failed-to-load-a-required-fdb-api-function-2204/4404 "2025-06-12T03:41:55Z")
I am upgrading some tooling from FoundationDB 7.2 to 7.3, and I am having this error with fdbcli: ERROR: Failed to load a required FDB API function. (2204) It works however if I call fdbcli --api-version 720 instead of… --- ## [\[Bugs\] DR with multiple dr\_agent](https://forums.foundationdb.org/t/bugs-dr-with-multiple-dr-agent/4934)
**Author:** [@hikouki-gumo](https://forums.foundationdb.org/u/hikouki-gumo)\ **Replies:** 0\ **Last updated:** [June 7, 2025, 3:35am UTC](https://forums.foundationdb.org/t/bugs-dr-with-multiple-dr-agent/4934 "2025-06-07T03:35:38Z")
Just ran into a pretty interesting incident with our FoundationDB DR cluster, and I want to dig deep into how fdb\_dr works internally after this. For us, the DR cluster is just as critical as the primary. Reproduce S… --- ## [Backup causing a lot of Data Movement on write heavy load](https://forums.foundationdb.org/t/backup-causing-a-lot-of-data-movement-on-write-heavy-load/4917)
**Author:** [@mpatou\_openai](https://forums.foundationdb.org/u/mpatou_openai)\ **Replies:** 1\ **Last updated:** [May 27, 2025, 2:24am UTC](https://forums.foundationdb.org/t/backup-causing-a-lot-of-data-movement-on-write-heavy-load/4917 "2025-05-27T02:24:31Z")
We conducted a test yesterday where we were reading 1GByte/s and reading 80MBytes/s on a 100 storage server clusters. The status json kept on showing that we were limited by log\_server\_write\_queue for batch performance … --- ## [Backup stats empty in status json](https://forums.foundationdb.org/t/backup-stats-empty-in-status-json/4916)
**Author:** [@czephyr](https://forums.foundationdb.org/u/czephyr)\ **Replies:** 1\ **Last updated:** [May 26, 2025, 1:25pm UTC](https://forums.foundationdb.org/t/backup-stats-empty-in-status-json/4916 "2025-05-26T13:25:31Z")
I’m trying to figure out why the status json output section for backups is not reporting stats in my setup. This is true for both 6.3 and 7.3 clusters. I have a machine running all backup\_agents and no fdbserver process… --- ## [Bulkload Performance Testing](https://forums.foundationdb.org/t/bulkload-performance-testing/4858)
**Author:** [@Northwo](https://forums.foundationdb.org/u/Northwo)\ **Replies:** 12\ **Last updated:** [May 26, 2025, 2:34am UTC](https://forums.foundationdb.org/t/bulkload-performance-testing/4858 "2025-05-26T02:34:35Z")
Really excited to see this bulkload feature released!, I’m currently testing on a cluster with 4 SSD nodes and 8 processes（1stateless + 1transaction + 6storage）. fdb\> status Using cluster file \`/etc/foundationdb/fdb.cl… --- ## [Fairly large time to become available after fallback in a multi region FDB](https://forums.foundationdb.org/t/fairly-large-time-to-become-available-after-fallback-in-a-multi-region-fdb/4915)
**Author:** [@mpatou\_openai](https://forums.foundationdb.org/u/mpatou_openai)\ **Replies:** 0\ **Last updated:** [May 22, 2025, 9:18pm UTC](https://forums.foundationdb.org/t/fairly-large-time-to-become-available-after-fallback-in-a-multi-region-fdb/4915 "2025-05-22T21:18:43Z")
Yesterday we ran a test for failover and fallback of FDB, failover went fairly smoothly. But fallback not as much. We had a setup with 2 FDB regions each with a main (where you have the whole set of roles/processes: st… --- ## [Non uniform out of blue regression in latency in GRV proxies](https://forums.foundationdb.org/t/non-uniform-out-of-blue-regression-in-latency-in-grv-proxies/4914)
**Author:** [@mpatou\_openai](https://forums.foundationdb.org/u/mpatou_openai)\ **Replies:** 0\ **Last updated:** [May 22, 2025, 8:07pm UTC](https://forums.foundationdb.org/t/non-uniform-out-of-blue-regression-in-latency-in-grv-proxies/4914 "2025-05-22T20:07:02Z")
Hello all, yesterday we were doing a performance testing of our metadata DB that is built on top of a mutli region FDB and as we ramped after 20mins of stability around 330K QPS reads to 350K QPS reads the GRV latency s… --- ## [Changing all coordinators in one step?](https://forums.foundationdb.org/t/changing-all-coordinators-in-one-step/4863)
**Author:** [@czephyr](https://forums.foundationdb.org/u/czephyr)\ **Replies:** 2\ **Last updated:** [May 20, 2025, 9:36am UTC](https://forums.foundationdb.org/t/changing-all-coordinators-in-one-step/4863 "2025-05-20T09:36:27Z")
I’m migrating some machines which participate to an FDB cluster. I will phase out the old 5 machines which host the coordinators and I will be required to change the coordinator roles to process running on the new machi… --- ## [Scaling log server and log to storage ratio](https://forums.foundationdb.org/t/scaling-log-server-and-log-to-storage-ratio/4890)
**Author:** [@mpatou\_openai](https://forums.foundationdb.org/u/mpatou_openai)\ **Replies:** 5\ **Last updated:** [May 15, 2025, 9:41pm UTC](https://forums.foundationdb.org/t/scaling-log-server-and-log-to-storage-ratio/4890 "2025-05-15T21:41:26Z")
I’m king of wondering on the impact of the scaling on Log servers, I was doing some load testing last week with 4 logs server per region and the log servers were clearly saturated on CPU (not on I/O it seems), so I quadr… --- ## [Recovery/Reviving a Storage Full Cluster](https://forums.foundationdb.org/t/recovery-reviving-a-storage-full-cluster/4909)
**Author:** [@msf](https://forums.foundationdb.org/u/msf)\ **Replies:** 1\ **Last updated:** [May 15, 2025, 8:58am UTC](https://forums.foundationdb.org/t/recovery-reviving-a-storage-full-cluster/4909 "2025-05-15T08:58:40Z")
Hi there, I’m doing some firefighting drills of “full cluster” scenarios using FDB 7.3.57 in a staging environment. Small test cluster: physical 3 nodes, 3 nvme drives per node for storage, fdb-logs and OS on separate… --- ## [Log processes and CPU saturation](https://forums.foundationdb.org/t/log-processes-and-cpu-saturation/4799)
**Author:** [@jon](https://forums.foundationdb.org/u/jon)\ **Replies:** 4\ **Last updated:** [May 15, 2025, 5:30am UTC](https://forums.foundationdb.org/t/log-processes-and-cpu-saturation/4799 "2025-05-15T05:30:31Z")
We’re performing some load tests with a new FoundationDB cluster (7.3.43 with three\_data\_hall replication and the Redwood storage engine). In this test, we scaled up load to the point where our log processes’ CPU consum… --- ## [Satellite\_logs not supported at the datacenter level in the kubernete operator?](https://forums.foundationdb.org/t/satellite-logs-not-supported-at-the-datacenter-level-in-the-kubernete-operator/4864)
**Author:** [@mpatou\_openai](https://forums.foundationdb.org/u/mpatou_openai)\ **Replies:** 1\ **Last updated:** [April 28, 2025, 12:39pm UTC](https://forums.foundationdb.org/t/satellite-logs-not-supported-at-the-datacenter-level-in-the-kubernete-operator/4864 "2025-04-28T12:39:14Z")
This page Configuration — FoundationDB ON documentation mentions that we can set satellite\_logs at both the datacenter and the region level but it seems after looking at 1.52 code base that the operator only supports the… --- ## [Multi region cluster, "regular" cluster elections: how to debug?](https://forums.foundationdb.org/t/multi-region-cluster-regular-cluster-elections-how-to-debug/4862)
**Author:** [@mpatou\_openai](https://forums.foundationdb.org/u/mpatou_openai)\ **Replies:** 4\ **Last updated:** [May 6, 2025, 1:28am UTC](https://forums.foundationdb.org/t/multi-region-cluster-regular-cluster-elections-how-to-debug/4862 "2025-05-06T01:28:04Z")
I have deployed 2 set of clusters that are using multiple regions and in both of them I’m seeing on a regular basis (ie. 3 to 5 times a day) issues with the cluster controller. When I look further on one cluster I notic… --- ## [FDB regular recovery with operator](https://forums.foundationdb.org/t/fdb-regular-recovery-with-operator/4562)
**Author:** [@jlemaes](https://forums.foundationdb.org/u/jlemaes)\ **Replies:** 16\ **Last updated:** [April 29, 2025, 11:07pm UTC](https://forums.foundationdb.org/t/fdb-regular-recovery-with-operator/4562 "2025-04-29T23:07:33Z")
We are migrating our FDB clusters from gcloud GCE instances to regional GKE with the k8s operator. The operator based clusters are working fine and really performant in our benchmarks. However, the new clusters on k8s h… --- ## [Looking for a Senior FoundationDB Developer in USA. It's a fully remote work with great compensation. Please help me out if you are interested or maybe suggest any of your friends or colleagues who you think might fit for this role](https://forums.foundationdb.org/t/looking-for-a-senior-foundationdb-developer-in-usa-its-a-fully-remote-work-with-great-compensation-please-help-me-out-if-you-are-interested-or-maybe-suggest-any-of-your-friends-or-colleagues-who-you-think-might-fit-for-this-role/4882)
**Author:** [@Abrar](https://forums.foundationdb.org/u/Abrar)\ **Replies:** 0\ **Last updated:** [April 29, 2025, 3:46pm UTC](https://forums.foundationdb.org/t/looking-for-a-senior-foundationdb-developer-in-usa-its-a-fully-remote-work-with-great-compensation-please-help-me-out-if-you-are-interested-or-maybe-suggest-any-of-your-friends-or-colleagues-who-you-think-might-fit-for-this-role/4882 "2025-04-29T15:46:41Z")
Looking for a FoundationDB Developer with 4+ years of experience. It’s fully remote work and the pay rate is also good. Experience with programming languages of C++, and AWS, Security, Privacy. You can reach me out or s… --- ## [Senior FoundationDB developer opportunity in USA (Remote Work)](https://forums.foundationdb.org/t/senior-foundationdb-developer-opportunity-in-usa-remote-work/4880)
**Author:** [@spicyb101](https://forums.foundationdb.org/u/spicyb101)\ **Replies:** 0\ **Last updated:** [April 29, 2025, 3:29pm UTC](https://forums.foundationdb.org/t/senior-foundationdb-developer-opportunity-in-usa-remote-work/4880 "2025-04-29T15:29:53Z")
If you are in or know any leads, kindly help me with it. --- ## [How to actually use FDB OpenTelemetry tracing?](https://forums.foundationdb.org/t/how-to-actually-use-fdb-opentelemetry-tracing/4877)
**Author:** [@ArniDagur](https://forums.foundationdb.org/u/ArniDagur)\ **Replies:** 1\ **Last updated:** [April 29, 2025, 5:16am UTC](https://forums.foundationdb.org/t/how-to-actually-use-fdb-opentelemetry-tracing/4877 "2025-04-29T05:16:18Z")
Is there anyone using OpenTelemetry tracing in production? If so, how is it configured? Would love a detailed example, since the docs are not helpful. I tried a simple example: First, create a fake trace collector list… --- ## [New feature bulkdump/bulkload using](https://forums.foundationdb.org/t/new-feature-bulkdump-bulkload-using/4833)
**Author:** [@fuCtor](https://forums.foundationdb.org/u/fuCtor)\ **Replies:** 4\ **Last updated:** [April 17, 2025, 7:55am UTC](https://forums.foundationdb.org/t/new-feature-bulkdump-bulkload-using/4833 "2025-04-17T07:55:31Z")
Hi! In FDB version 7.4.0 added new feature bulkdump/bulkload. I try use it, but it don’t work. For reproduce: \> configure new single ssd \> writemode on \> set foo bar \> bulkdump mode on \> bulkdump dump \\x00 \\xff /var/fd… --- ## [Unable to build a cluster of ubuntu vms](https://forums.foundationdb.org/t/unable-to-build-a-cluster-of-ubuntu-vms/4839)
**Author:** [@aria3ppp](https://forums.foundationdb.org/u/aria3ppp)\ **Replies:** 0\ **Last updated:** [April 13, 2025, 2:48pm UTC](https://forums.foundationdb.org/t/unable-to-build-a-cluster-of-ubuntu-vms/4839 "2025-04-13T14:48:29Z")
I cannot build a cluster of two foundationdb ubuntu vm instances the doc says i should copy my current fdb.cluster file content to the new ones: after that i should run fdbcli --exec "coordinators vm1:4500 vm2:4500" bu… --- ## [FDB v7 writing only to one disk out of five](https://forums.foundationdb.org/t/fdb-v7-writing-only-to-one-disk-out-of-five/4824)
**Author:** [@franciscilwin1](https://forums.foundationdb.org/u/franciscilwin1)\ **Replies:** 1\ **Last updated:** [April 13, 2025, 6:45pm UTC](https://forums.foundationdb.org/t/fdb-v7-writing-only-to-one-disk-out-of-five/4824 "2025-04-13T18:45:37Z")
Hi, We recently upgraded from FDB v6 to FDB v7. We’re running performance tests for this. We have a six node setup, and that setup has been replicated for the new fdb server because we’re also upgrading OS etc. What we’… --- ## [Smaller memory allocations for stateless pods?](https://forums.foundationdb.org/t/smaller-memory-allocations-for-stateless-pods/4797)
**Author:** [@jon](https://forums.foundationdb.org/u/jon)\ **Replies:** 2\ **Last updated:** [April 10, 2025, 9:12pm UTC](https://forums.foundationdb.org/t/smaller-memory-allocations-for-stateless-pods/4797 "2025-04-10T21:12:35Z")
We’re building out a new FoundationDB cluster, and I’m thinking about resource allocations after an initial load test. We’re using the FDB Kubernetes Operator and previously ran a load test that handled 100% of our write… --- ## [Fdbbackup continuous internal](https://forums.foundationdb.org/t/fdbbackup-continuous-internal/4777)
**Author:** [@hikouki-gumo](https://forums.foundationdb.org/u/hikouki-gumo)\ **Replies:** 3\ **Last updated:** [April 10, 2025, 5:51pm UTC](https://forums.foundationdb.org/t/fdbbackup-continuous-internal/4777 "2025-04-10T17:51:41Z")
Reference: foundationdb/design/backup.md at main · apple/foundationdb · GitHub. Hi, I’m running fdbbackup with option -z (continuous) with snapshot-interval = 1 days, for example, take a snapshot 01:00 daily and store t… --- ## [Why MaxRestoreableVersion not updates?](https://forums.foundationdb.org/t/why-maxrestoreableversion-not-updates/4822)
**Author:** [@1412904892](https://forums.foundationdb.org/u/1412904892)\ **Replies:** 2\ **Last updated:** [April 3, 2025, 3:51am UTC](https://forums.foundationdb.org/t/why-maxrestoreableversion-not-updates/4822 "2025-04-03T03:51:29Z")
I currently have a backup job. When I check it using fdbbackup describe , most snapshots show restorable=false . Does this mean these snapshots cannot be restored? However, after I run fdbrestore , the previously restora… --- ## [Rebalancing not happening on read hot shards in 7.4.43?](https://forums.foundationdb.org/t/rebalancing-not-happening-on-read-hot-shards-in-7-4-43/4821)
**Author:** [@mpatou\_openai](https://forums.foundationdb.org/u/mpatou_openai)\ **Replies:** 3\ **Last updated:** [April 2, 2025, 9:20pm UTC](https://forums.foundationdb.org/t/rebalancing-not-happening-on-read-hot-shards-in-7-4-43/4821 "2025-04-02T21:20:45Z")
Over the course of the week-end one of our cluster is starting to see a lot more load and some of the storage servers are using ~75% of CPU on average when some others are mostly idle. I already mentioned this a bit in … --- ## [Fdbbackup version and snapshot and their relationship to restore](https://forums.foundationdb.org/t/fdbbackup-version-and-snapshot-and-their-relationship-to-restore/4112)
**Author:** [@tangerine](https://forums.foundationdb.org/u/tangerine)\ **Replies:** 1\ **Last updated:** [April 1, 2025, 6:53am UTC](https://forums.foundationdb.org/t/fdbbackup-version-and-snapshot-and-their-relationship-to-restore/4112 "2025-04-01T06:53:55Z")
Hi, I have some questions regarding backup and restore. I run continuous backup in my cluster and it is setup to create a new snapshot every 10 days and incremental in between. Also, I run fdbbackup expire to expire… --- ## [How accurate is readOpsPerSec when readSample is enabled?](https://forums.foundationdb.org/t/how-accurate-is-readopspersec-when-readsample-is-enabled/4818)
**Author:** [@mpatou\_openai](https://forums.foundationdb.org/u/mpatou_openai)\ **Replies:** 0\ **Last updated:** [March 31, 2025, 5:39pm UTC](https://forums.foundationdb.org/t/how-accurate-is-readopspersec-when-readsample-is-enabled/4818 "2025-03-31T17:39:41Z")
I’m wondering how reliable is this metric in 7.3.43 because I was looking at two storage servers yesterday and they showed very different CPU consumption (ie. 75% vs ~5%) but had readOpsPerSec somewhat similar: Low CPU: … --- ## [FDB 8.0 release time](https://forums.foundationdb.org/t/fdb-8-0-release-time/4801)
**Author:** [@Northwo](https://forums.foundationdb.org/u/Northwo)\ **Replies:** 2\ **Last updated:** [March 31, 2025, 1:44am UTC](https://forums.foundationdb.org/t/fdb-8-0-release-time/4801 "2025-03-31T01:44:04Z")
When will FDB 8.0 be released, and when will the documentation be updated? --- ## [UnableToWriteStatus error in backup-agent when number of backups grows large](https://forums.foundationdb.org/t/unabletowritestatus-error-in-backup-agent-when-number-of-backups-grows-large/4810)
**Author:** [@sighol](https://forums.foundationdb.org/u/sighol)\ **Replies:** 1\ **Last updated:** [March 29, 2025, 6:22pm UTC](https://forums.foundationdb.org/t/unabletowritestatus-error-in-backup-agent-when-number-of-backups-grows-large/4810 "2025-03-29T18:22:51Z")
We are running weekly backups of our FDB clusters. In our most long-lived clusters, there is now around 180 backups under the \\xff\\x02/backup-agent/tag-\>uid/prefix. For the biggest clusters, we are observing that the ba… [Previous page](https://forums.foundationdb.org/c/using-foundationdb/7.md?page=2) [Next page](https://forums.foundationdb.org/c/using-foundationdb/7.md?page=4)