# FoundationDBBackup backup command error

**URL:** <https://forums.foundationdb.org/t/foundationdbbackup-backup-command-error/2341>\
**Category:** Kubernetes Operator\
**Created:** [September 17, 2020, 8:19pm UTC](https://forums.foundationdb.org/t/foundationdbbackup-backup-command-error/2341 "2020-09-17T20:19:07Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![mbuleandra](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/mbuleandra/32/977_2.png) [@mbuleandra](https://forums.foundationdb.org/u/mbuleandra)\
**Post date:** [September 17, 2020, 8:19pm UTC](https://forums.foundationdb.org/t/foundationdbbackup-backup-command-error/2341/1 "2020-09-17T20:19:07Z")

</div>

The default fdbbackup command fails for the FoundationDBBackup resource.  
Running the same command in a shell in a backup pod will also result in an error, unless the URL parameter is surrounded by double quotes. I couldn’t find a way to inject the double quotes using the FoundationDBBackup configuration.

```
apiVersion: apps.foundationdb.org/v1beta1
kind: FoundationDBBackup
metadata:
  name: storage-cluster
spec:
  version: 6.2.20
  clusterName: storage-cluster
  backupName: storage-cluster-auto
  bucket: fdb-backups&sc=0
  accountName: key@s3.us-gov-west-1.amazonaws.com:80
  snapshotPeriodSeconds: 60

```

Command Fails:

```
fdbbackup start -d blobstore://key@s3.us-gov-west-1.amazonaws.com:80/storage-cluster-auto?bucket=fdb-backups&sc=0

```

Command Succeeds: (added double quotes surrounding the URL param)

```
fdbbackup start -d "blobstore://key@s3.us-gov-west-1.amazonaws.com:80/storage-cluster-auto?bucket=fdb-backups&sc=0"

```

Error in the operator:

```
2020-09-17T19:39:10.283Z	INFO	controller	Running command	{"namespace": "default", "cluster": "storage-cluster", "path": "/usr/bin/fdb/6.2/fdbbackup", "args": ["/usr/bin/fdb/6.2/fdbbackup", "status", "--json", "-C", "/tmp/483506077", "--log", "--logdir", "/var/log/fdb"]}
2020-09-17T19:39:10.339Z	INFO	controller	Command completed	{"namespace": "default", "cluster": "storage-cluster", "output": "{\"SchemaVersion\":\"1...."}
2020-09-17T19:39:10.339Z	INFO	controller	Running command	{"namespace": "default", "cluster": "storage-cluster", "path": "/usr/bin/fdb/6.2/fdbbackup", "args": ["/usr/bin/fdb/6.2/fdbbackup", "start", "-d", "blobstore://key@s3.us-gov-west-1.amazonaws.com:80/storage-cluster-auto?bucket=fdb-backups&sc=0", "-s", "60", "-z", "-C", "/tmp/023463512", "--log", "--logdir", "/var/log/fdb"]}
2020-09-17T19:39:20.371Z	ERROR	controller	Error from FDB command	{"namespace": "default", "cluster": "storage-cluster", "code": -1, "stdout": "", "stderr": "", "error": "signal: killed"}
github.com/go-logr/zapr.(*zapLogger).Error
	/go/pkg/mod/github.com/go-logr/zapr@v0.1.0/zapr.go:128
github.com/FoundationDB/fdb-kubernetes-operator/controllers.(*CliAdminClient).runCommand
	/workspace/controllers/admin_client.go:255
github.com/FoundationDB/fdb-kubernetes-operator/controllers.(*CliAdminClient).StartBackup
	/workspace/controllers/admin_client.go:509
github.com/FoundationDB/fdb-kubernetes-operator/controllers.StartBackup.Reconcile
	/workspace/controllers/start_backup.go:46
github.com/FoundationDB/fdb-kubernetes-operator/controllers.(*FoundationDBBackupReconciler).Reconcile
	/workspace/controllers/backup_controller.go:82
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.4.0/pkg/internal/controller/controller.go:256
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.4.0/pkg/internal/controller/controller.go:232
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.4.0/pkg/internal/controller/controller.go:211
k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1
	/go/pkg/mod/k8s.io/apimachinery@v0.17.0/pkg/util/wait/wait.go:152
k8s.io/apimachinery/pkg/util/wait.JitterUntil
	/go/pkg/mod/k8s.io/apimachinery@v0.17.0/pkg/util/wait/wait.go:153
k8s.io/apimachinery/pkg/util/wait.Until
	/go/pkg/mod/k8s.io/apimachinery@v0.17.0/pkg/util/wait/wait.go:88

```

Error from the fdbbackup logs:

```
<Event Severity="10" Time="1600371759.325817" Type="FBA_SubmitBackup" ID="0000000000000000" TagName="default" StopWhenDone="1" OutContainer="blobstore://key@s3.us-gov-west-1.amazonaws.com:80/storage-cluster-auto?bucket=fdb-backups" Machine="192.168.77.17:94" LogGroup="default" />
<Event Severity="20" Time="1600371769.349597" Type="BlobStoreEndpointRequestFailedRetryable" ID="0000000000000000" Error="timed_out" ErrorDescription="Operation timed out" ErrorCode="1004" SuppressedEventCount="0" ConnectionEstablished="0" RemoteHost="s3.us-gov-west-1.amazonaws.com" Verb="HEAD" Resource="/fdb-backups" ThisTry="1" RetryDelay="2" Machine="192.168.77.17:94" LogGroup="default" />
<Event Severity="40" Time="1600371789.329575" Type="StopAfterError" ID="0000000000000000" Error="backup_error" ErrorDescription="Backup error" ErrorCode="2300" Backtrace="addr2line -e fdbbackup.debug -p -C -f -i 0xaa654c 0xaa5e68 0xaa5f31 0x4d4ed3 0x4d50be 0x4f6339 0x4db605 0x4dbc3e 0x4f6339 0x4d03aa 0x4d0700 0x4f6339 0x699121 0x699430 0x4f6339 0x4ca230 0x4f7bb0 0xaeac50 0x83c8b2 0x4b1469 0x7f2f829afb97" Machine="192.168.77.17:94" LogGroup="default" />
```

---

<div class="post-metadata">

**Author:** ![john\_brownlee](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/john_brownlee/32/22_2.png) [@john\_brownlee](https://forums.foundationdb.org/u/john_brownlee)\
**Post date:** [September 17, 2020, 8:32pm UTC](https://forums.foundationdb.org/t/foundationdbbackup-backup-command-error/2341/2 "2020-09-17T20:32:58Z")

</div>

With the way we’re executing the commands from the operator, I wouldn’t expect any kind of quotation to be necessary, because it’s not going through a shell.

---

<div class="post-metadata">

**Author:** ![mbuleandra](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/mbuleandra/32/977_2.png) [@mbuleandra](https://forums.foundationdb.org/u/mbuleandra)\
**Post date:** [September 17, 2020, 8:51pm UTC](https://forums.foundationdb.org/t/foundationdbbackup-backup-command-error/2341/3 "2020-09-17T20:51:34Z")

</div>

Is there any info I could collect to help you debug this?  
It seems that running from the shell works, but the operator command fails

---

<div class="post-metadata">

**Author:** ![john\_brownlee](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/john_brownlee/32/22_2.png) [@john\_brownlee](https://forums.foundationdb.org/u/john_brownlee)\
**Post date:** [September 17, 2020, 8:54pm UTC](https://forums.foundationdb.org/t/foundationdbbackup-backup-command-error/2341/4 "2020-09-17T20:54:49Z")

</div>

Could you try running the command from a shell in the operator pod? There may be some differences in the environment between the operator and the backup agents.

---

<div class="post-metadata">

**Author:** ![john\_brownlee](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/john_brownlee/32/22_2.png) [@john\_brownlee](https://forums.foundationdb.org/u/john_brownlee)\
**Post date:** [September 17, 2020, 8:58pm UTC](https://forums.foundationdb.org/t/foundationdbbackup-backup-command-error/2341/5 "2020-09-17T20:58:51Z")

</div>

I modified the minio instance in my dev environment to run on a non-TLS connection, and modified the cluster\_with\_backup example to use the bucket name `fdb-backups&sc=0`. It was able to start the backup and complete reconciliation. This suggests the problem is something other than the additional parameters.