# \`fdbbackup\` with TLS not working

**URL:** <https://forums.foundationdb.org/t/fdbbackup-with-tls-not-working/3323>\
**Category:** Kubernetes Operator\
**Created:** [May 16, 2022, 7:04am UTC](https://forums.foundationdb.org/t/fdbbackup-with-tls-not-working/3323 "2022-05-16T07:04:49Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![mason](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/mason/32/1145_2.png) [@mason](https://forums.foundationdb.org/u/mason)\
**Post date:** [May 16, 2022, 7:04am UTC](https://forums.foundationdb.org/t/fdbbackup-with-tls-not-working/3323/1 "2022-05-16T07:04:49Z")

</div>

Hi all,  
I setup FDB cluster by FDBCluster operator with TLS (self-signed certificate by `cert-manager`). Then I setup FDBBackup that use same TLS certificate. After backup deployment started, only `backup_agent` process running inside backup pod, `fdbbackup` process is not (1). Then I run `fdbbackup start` manual, if I use parameter `sc=0` (disable TLS) the backup work, else `sc=1` (enable TLS) it not (2).

I have 2 point unclear. First is as FDB backup document, after setup FDB backup by operator, it will start 2 process are `backup_agent` and `fdbbackup start`, in my case only `backup_agent`. Second is TLS is used and worked for communicate inside cluster only, not work for talk to S3.  
Here is my config for FDBBackup operator

```auto
apiVersion: apps.foundationdb.org/v1beta2
kind: FoundationDBBackup
metadata:
  name: fdb-cluster
spec:
  blobStoreConfiguration:
    accountName: <access_key_id>@s3.us-east-2.amazonaws.com
    backupName: daily-backup
    bucket: ci-cd-code-deploy
  clusterName: fdb-cluster
  customParameters:
    - "knob_http_request_aws_v4_header=true"
    - "knob_http_verbose_level=4"
  podTemplateSpec:
    spec:
      containers:
        - env:
          - name: FDB_BLOB_CREDENTIALS
            value: /var/backup-credentials/credentials
          - name: FDB_TLS_CERTIFICATE_FILE
            value: /var/fdb-certs/tls.crt
          - name: FDB_TLS_CA_FILE
            value: /var/trust-ca/ca.crt
          - name: FDB_TLS_KEY_FILE
            value: /var/fdb-certs/tls.key
          name: foundationdb
          resources:
            limits:
              cpu: 250m
              memory: 128Mi
            requests:
              cpu: 250m
              memory: 128Mi
          securityContext:
            runAsGroup: 0
            runAsUser: 0
          volumeMounts:
            - mountPath: /var/fdb-certs
              name: fdb-certs
            - mountPath: /var/trust-ca
              name: trust-ca
            - mountPath: /var/backup-credentials
              name: backup-credentials
      initContainers:
        - name: foundationdb-kubernetes-init
          resources:
            limits:
              cpu: 100m
              memory: 128Mi
            requests:
              cpu: 100m
              memory: 128Mi
          securityContext:
            runAsUser: 0
      volumes:
        - name: backup-credentials
          secret:
            secretName: aws-credentials
        - name: fdb-certs
          secret:
            secretName: fdb-cluster-certificate
        - name: trust-ca
          secret:
            secretName: root-secret
  snapshotPeriodSeconds: 60
  version: 6.3.23

```

---

<div class="post-metadata">

**Author:** ![johscheuer](https://avatars.discourse-cdn.com/v4/letter/j/f475e1/32.png) [@johscheuer](https://forums.foundationdb.org/u/johscheuer)\
**Post date:** [May 17, 2022, 6:32am UTC](https://forums.foundationdb.org/t/fdbbackup-with-tls-not-working/3323/2 "2022-05-17T06:32:59Z")

</div>

For your questions:

1.) The `fdbbackup start` command will be triggered by the operator itself and the backup\_agent(s) will then to the actual backup work.  
2.) TLS is used for communicating the the cluster and S3 and you have to ensure that the provided CA file has all required root CAs (the one for your cluster and the root CA for S3).

When you take a look at the trace files in the operator you’ll probably see some TLS verification issues they are located by default under `/var/log/fdb`.

---

<div class="post-metadata">

**Author:** ![mason](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/mason/32/1145_2.png) [@mason](https://forums.foundationdb.org/u/mason)\
**Post date:** [May 17, 2022, 6:41am UTC](https://forums.foundationdb.org/t/fdbbackup-with-tls-not-working/3323/3 "2022-05-17T06:41:11Z")

</div>

Hi @johscheuer ,

1. My understand is right ( `fdbbackup start` command will be triggered by the operator but it doesn’t work in my case)
2. I read the log, it responses

```auto
<Event Severity="20" Time="1652766871.043128" DateTime="2022-05-17T05:54:31Z" Type="TLSPolicyFailure" ID="0000000000000000" SuppressedEventCount="0" Reason="preverification failed" VerifyError="unable to get local issuer certificate" ThreadID="13788610134641559662" Machine="10.42.1.46:143" LogGroup="default" />
<Event Severity="20" Time="1652766871.043128" DateTime="2022-05-17T05:54:31Z" Type="N2_ConnectHandshakeError" ID="6fa4c5f1a20071f6" SuppressedEventCount="0" ErrorCode="337047686" Message="certificate verify failed" WhichMeans="error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed" ThreadID="13788610134641559662" Machine="10.42.1.46:143" LogGroup="default" />

```

My question is can I use self signed certificate for this? And if yes, how can I create certificate that sufficient as your answer

```auto
TLS is used for communicating the the cluster and S3 and you have to ensure that the provided CA file has all required root CAs (the one for your cluster and the root CA for S3)

```

---

<div class="post-metadata">

**Author:** ![johscheuer](https://avatars.discourse-cdn.com/v4/letter/j/f475e1/32.png) [@johscheuer](https://forums.foundationdb.org/u/johscheuer)\
**Post date:** [May 17, 2022, 7:04am UTC](https://forums.foundationdb.org/t/fdbbackup-with-tls-not-working/3323/4 "2022-05-17T07:04:05Z")

</div>

Is your cluster running with the same certificates? The following docs have more details about the TLS requirements: [Transport Layer Security — FoundationDB 7.1](https://apple.github.io/foundationdb/tls.html#required-files). Yes you can do that, you just have to ensure that the CA file contains the root CA for your self-singed certificate and the root CA’s for S3.

---

<div class="post-metadata">

**Author:** ![mason](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/mason/32/1145_2.png) [@mason](https://forums.foundationdb.org/u/mason)\
**Post date:** [May 17, 2022, 7:16am UTC](https://forums.foundationdb.org/t/fdbbackup-with-tls-not-working/3323/5 "2022-05-17T07:16:47Z")

</div>

Hi @johscheuer , I use same certificates (self signed) for cluster and use that CA for TLS communicate with S3 also. So now I need to create new certificate for S3? And root CA for S3 must be signed by third party like `letencrypt` or self signed is acceptable?

---

<div class="post-metadata">

**Author:** ![johscheuer](https://avatars.discourse-cdn.com/v4/letter/j/f475e1/32.png) [@johscheuer](https://forums.foundationdb.org/u/johscheuer)\
**Post date:** [May 18, 2022, 7:45am UTC](https://forums.foundationdb.org/t/fdbbackup-with-tls-not-working/3323/6 "2022-05-18T07:45:08Z")

</div>

You don’t need to create a new certificate for S3 Amazon already have their own certificates, you just have to download the root CA for S3 from Amazon, see: [Amazon Trust Services Repository](https://www.amazontrust.com/repository)

---

<div class="post-metadata">

**Author:** ![mason](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/mason/32/1145_2.png) [@mason](https://forums.foundationdb.org/u/mason)\
**Post date:** [May 18, 2022, 8:01am UTC](https://forums.foundationdb.org/t/fdbbackup-with-tls-not-working/3323/7 "2022-05-18T08:01:59Z")

</div>

Hi @johscheuer , after download root CA from Amazon, I need to append Amazon root CA to my CA, like:

```auto
----------BEGIN---------
Here is my CA
----------END------------
----------BEGIN---------
Here is Amazon CA
----------END------------

```