FoundationDB backup s3 https connection_failed

I’m trying to write a backup to AWS S3, via HTTPS, but I get a connection_failed message:

fdbbackup start -d "blobstore://key:secret@s3.us-gov-west-1.amazonaws.com:443/myback?bucket=fdb-backups&sc=1"

Log File:

<Event Severity="10" Time="1600297291.472883" Type="FBA_SubmitBackup" ID="0000000000000000" TagName="default" StopWhenDone="1" OutContainer="blobstore://key:secret@s3.us-gov-west-1.amazonaws.com:443/myback?bucket=fdb-backups&amp;sc=1" Machine="192.168.69.235:633" LogGroup="default" />
<Event Severity="20" Time="1600297291.500330" Type="BlobStoreEndpointRequestFailedRetryable" ID="0000000000000000" Error="connection_failed" ErrorDescription="Network connection failed" ErrorCode="1026" SuppressedEventCount="0" ConnectionEstablished="0" RemoteHost="s3.us-gov-west-1.amazonaws.com" Verb="HEAD" Resource="/fdb-backups" ThisTry="1" RetryDelay="2" Machine="192.168.69.235:633" LogGroup="default" />

Running on HTTP works:

fdbbackup start -d "blobstore://key:secret@s3.us-gov-west-1.amazonaws.com:80/myback?bucket=fdb-backups&sc=0"

Do you have any suggestions on how to work around this issue?

I have tried other combinations of setting the sc parameter and setting a port or setting no port, but without success. More so, if I forget to set the port :80 for the HTTP blobstore URL, I get a misleading DNS lookup fail message:

<Event Severity="20" Time="1600294151.725271" Type="BlobStoreEndpointRequestFailedRetryable" ID="0000000000000000" Error="lookup_failed" ErrorDescription="DNS lookup failed" ErrorCode="1041" SuppressedEventCount="0" ConnectionEstablished="0" RemoteHost="s3.us-gov-west-1.amazonaws.com" Verb="HEAD" Resource="/fdb-backups" ThisTry="1" RetryDelay="2" Machine="192.168.69.235:476" LogGroup="default" />

Version:

fdbbackup list --version
FoundationDB 6.2 (v6.2.20)
source version 77b5171e81754f2fda8869703d662e59d85b7f23
protocol fdb00b062010001
1 Like

Hey, any insight here? Basically just trying to get backups to work in S3 over HTTPS

Check your logs for events involving “TLS”.

My guess is you have not configured the TLS certificate, private key, and trusted certificate authorities. I do not see the options in your command line, but they can also be configured via these environment variables:

FDB_TLS_CERTIFICATE_FILE
FDB_TLS_KEY_FILE
FDB_TLS_CA_FILE

For more information, see here:

Thanks, that did it!
I’m using a self-signed certificate so I had to disable peer verification on the backup agent

You can avoid having to disable verification by creating a CA file to use with FDB that is the concatenation of your self-signed certificate file contents and the contents of a CA file of typical roots that browsers and operating systems trust. Possible locations a system default CA file are:

/usr/local/etc/openssl/cert.pem
/etc/ssl/certs/ca-certificates.crt

But it is system/distribution dependent.