Scripts to spin up FoundationDB cluster on AWS

Using FoundationDB
1

Here are packer and terraform scripts that that you can use to run cost-effective experiments with large FoundationDB clusters on AWS: github.com/bitgn/fdb-cloud-test.

The process involves the following steps:

  1. Create AMI images for your setup, installing necessary software. This is usually done once.
  2. Use terraform to spin up FoundationDB and tester machines out of these AMI images.
  3. Run experiments and benchmarks
  4. Tear down the cluster

Once the AMI images is “baked” and added to the AWS account, it usually takes 3-4 minutes to create a cluster and a few minutes to tear it down. Since AWS charges per second, you can have experiment even with a large cluster without spending a lot of money.

I use similar scripts to benchmark performance of FoundationDB layers in various configurations.

4 Likes
2

Updates:

  1. Support for launching multiple FoundationDB processes on the new cluster
  2. m3.large, m3.medium and i3.large machines properly mount instance store and use it for storing FoundationDB data
  3. Tester AMI packs mosh and opens appropriate ports

This was tested with the clusters up to 15 x i3.large VMs

3

hey @abdullin , there is another repo which is quite similar to your solution, I tried that on AWS and run several RandomReadWrite tests on the cluster, but I found the latency is quite bigger than the official performance post, I added the result in detail there, do you know which reason leads to the difference?