FoundationDB backup s3 https connection_failed

Using FoundationDB
1

I’m trying to write a backup to AWS S3, via HTTPS, but I get a connection_failed message:

fdbbackup start -d "blobstore://key:secret@s3.us-gov-west-1.amazonaws.com:443/myback?bucket=fdb-backups&sc=1"

Log File:

<Event Severity="10" Time="1600297291.472883" Type="FBA_SubmitBackup" ID="0000000000000000" TagName="default" StopWhenDone="1" OutContainer="blobstore://key:secret@s3.us-gov-west-1.amazonaws.com:443/myback?bucket=fdb-backups&amp;sc=1" Machine="192.168.69.235:633" LogGroup="default" />
<Event Severity="20" Time="1600297291.500330" Type="BlobStoreEndpointRequestFailedRetryable" ID="0000000000000000" Error="connection_failed" ErrorDescription="Network connection failed" ErrorCode="1026" SuppressedEventCount="0" ConnectionEstablished="0" RemoteHost="s3.us-gov-west-1.amazonaws.com" Verb="HEAD" Resource="/fdb-backups" ThisTry="1" RetryDelay="2" Machine="192.168.69.235:633" LogGroup="default" />

Running on HTTP works:

fdbbackup start -d "blobstore://key:secret@s3.us-gov-west-1.amazonaws.com:80/myback?bucket=fdb-backups&sc=0"

Do you have any suggestions on how to work around this issue?

I have tried other combinations of setting the sc parameter and setting a port or setting no port, but without success. More so, if I forget to set the port :80 for the HTTP blobstore URL, I get a misleading DNS lookup fail message:

<Event Severity="20" Time="1600294151.725271" Type="BlobStoreEndpointRequestFailedRetryable" ID="0000000000000000" Error="lookup_failed" ErrorDescription="DNS lookup failed" ErrorCode="1041" SuppressedEventCount="0" ConnectionEstablished="0" RemoteHost="s3.us-gov-west-1.amazonaws.com" Verb="HEAD" Resource="/fdb-backups" ThisTry="1" RetryDelay="2" Machine="192.168.69.235:476" LogGroup="default" />

Version:

fdbbackup list --version
FoundationDB 6.2 (v6.2.20)
source version 77b5171e81754f2fda8869703d662e59d85b7f23
protocol fdb00b062010001
1 Like
2

Hey, any insight here? Basically just trying to get backups to work in S3 over HTTPS

3

Check your logs for events involving “TLS”.

My guess is you have not configured the TLS certificate, private key, and trusted certificate authorities. I do not see the options in your command line, but they can also be configured via these environment variables:

FDB_TLS_CERTIFICATE_FILE
FDB_TLS_KEY_FILE
FDB_TLS_CA_FILE

For more information, see here:

4

Thanks, that did it!
I’m using a self-signed certificate so I had to disable peer verification on the backup agent

5

You can avoid having to disable verification by creating a CA file to use with FDB that is the concatenation of your self-signed certificate file contents and the contents of a CA file of typical roots that browsers and operating systems trust. Possible locations a system default CA file are:

/usr/local/etc/openssl/cert.pem
/etc/ssl/certs/ca-certificates.crt

But it is system/distribution dependent.

6

Hi @SteavedHams ,

Can I use self signed certificate for both TLS in cluster and S3 backup?