Fdbbackup: backup_error

Using FoundationDB
1

Hello, I’m new to foundationdb as well as S3 and wanted to perform the backup to aws S3. I am using the command : fdbbackup start -w -t “tag1” -d blobstore://<api_key>[:]@s3.ap-south-1.amazonaws.com:80/<backup_file_name>?bucket=<bucket_name>&sc=0

while trying this I get an HTTP 400 bad request, unsure where I’m missing out.
output after --knob_http_verbose_level = 3,

image
image1297×455 81.4 KB

I checked the trace.xml files as well for the backup :

Screenshot from 2021-03-26 15-37-37
Screenshot from 2021-03-26 15-37-371372×324 106 KB

additional info:

  • I have the cluster running on one single node and foundationdb version is 6.2.19
  • Cluster has no TLS enabled
  • S3 user is configured to have full access to the objects in the bucket.

additonal clarification, the would be the S3 user’s AWS_ACCESS_KEY_ID, and the would be the AWS_SECRET_ACCESS_KEY right?

Could anyone help me figure out how to resolve this?

2

FoundationdB cannot be used with S3 (for buckets created after June 2020) till it supports AWS Signature Version 4. The older one (v2) was deprecated on June 2020.

Further reading:

3

Given the issue mentioned above, is there a workaround for this? @alloc @SteavedHams

4

I don’t believe there is (if one still wants to use s3 and one doesn’t have an older bucket). I think the only solution if one wants to use a new (i.e., created after June 2020) s3 bucket, would be for that issue, #2907, to be completed.

5

But I’m also not an expert on this, so it’s possible someone who knows more about s3 could correct me if I’m wrong.

6

You can use Minio to both get around this specific issue and to abstract over storage backends if you’re running across multiple cloud providers.

7

Hi Folks, the issue regarding backup got resolved when I switched my bucket region from ap-south-1 to us-east-1. I am unsure as to why this change got the backups to work, Can anyone shed light on the differences between the regions? Also, how does foundationdb backup interprets the blobstore URL and where does the signature version play an authentication role?

Edit: tried with a bunch of other regions and got the same backup_error ( HTTP response code failed )

8

Hi, I hit this a few months ago. It works on us-east-1 because V2 is still enabled on regions created before Jan 30 2014. (I think general rule of thumb is it works in US regions only)

I run FDB in a wrapper service, in non-US deployments I configured it to take the backup locally, then the wrapper service zips the backup files and uploads it to S3.