Dockerized deployment

Using FoundationDB
1

I’ve noticed that there is an official foundationdb Docker image. That’s fantastic, as Docker support is essential for many of us! But I have some questions and suggestions:

  • can we rely on this docker image to be maintained and updated?
  • the image seems to lack a README, even including the README that is in the source code next to the Dockerfile would be helpful!
  • I am looking at https://github.com/apple/foundationdb/issues/1730 and just wanted to point out that local development testing is not the only use case: deployment is, too, as well as developing with pre-populated data, so the ability to mount a volume as the data directory seems essential.
  • it took me a long time to stumble onto the example at https://github.com/apple/foundationdb/tree/master/packaging/docker/samples/local — perhaps this could be added to the README?
  • the example requires a local (host-native) fdbcli, wouldn’t it be better to use docker run using fdbcli from the image?

Lastly, on a more general note, browsing through GitHub issues I see mentions of Kubernetes deployments. I’d say there are several general classes of use cases for docker images:

  • local development/testing (e.g. docker run, docker-compose locally, with a docker-compose network)
  • remote development/testing (same thing, but executed using docker-machine on remote servers)
  • static dockerized deployment (docker-compose or simple docker run, but in production)
  • orchestrated deployment using simple tools (nomad, docker swarm)
  • Kubernetes/OpenStack

I wanted to point this out, because a well-designed docker image is useful to all of these use cases.

Also, while I am personally not a fan of Kubernetes and its complexity, I know that in a number of real-life enterprise scenarios it has become a hard requirement: a vendor selling a solution must be able to deploy it on customer-provided Kubernetes infrastructure. Which means that dockerization work is important for FoundationDB adoption.

1 Like
2

Updating it as a part of the release process, so it should be updated in coordination with downloads being posted to the website.

For the suggestions, you are welcome to open issues for the docker improvements that you’d like to see. :slight_smile:

This is well known. The struggle is that having a well-designed docker image that is useful to a large number of use-cases requires a person doing the work that knows how to design a docker image well that will satisfy a large number of use-cases. This is something that, I believe, the current set of companies sponsoring FDB work lack.

I’ve been hopeful that this is an area where the community would help define or create the appropriate solution. If anyone would like to post a design doc for what a well-designed docker image would look like, it’s overly welcome. Early in FDB’s OSS history, there was an initial community PR that tried to do this, and the work stalled out for a variety of reasons. I think it highlights why we likely haven’t seen continued progress, unfortunately. The current docker image was meant as the smallest step forward that still yielded a meaningful improvement.

My current hope has been that from the upcoming FDB summit, some folk will give talks and release what they’ve done for Kubernetes/docker, and that will help to push the area along.

4 Likes
3

Thanks, Alex, this is great to hear!

As for filing issues or helping, I don’t feel qualified enough to do that, as I’m not a Docker expert, and I’m just learning about FoundationDB. There might have been reasons for some decisions, which I do not understand. For example, the FDB docker container doesn’t EXPOSE any ports, and some examples use host networking. In general, the idea behind containerization is that you only use the ports that you exposed and avoid host networking.

Another thing: FDB seems to make a lot of assumptions and actually implements some discovery/coordination/orchestration features on its own. The cluster file is assumed to be the same for all processes connecting to the cluster, which I don’t think is desirable, or even possible with dockerized deployments. Also, the cluster file uses IP addresses, while in docker you’d rather see names, which get resolved through docker/docker-compose, consul, etcd or other discovery services.

Then there is fdbmonitor: I’m not sure what is the right granularity for a dockerized FDB. If we want to go down to single-process level, then fdbmonitor is not desirable. I think it makes more sense to deploy FDB “units”, not individual processes. This leaves open the question of container health checks.

As I wrote, I do not feel qualified to provide answers and solutions at this point, I just wanted to know if containerization is considered important. I’m very glad to hear that it is and I will try to contribute as much as I can.

1 Like
4

local development/testing (e.g. docker run, docker-compose locally, with a docker-compose network)

Does anyone have other examples of getting foundationdb setup using docker-compse? I’ve been using the packaging/docker/samples as a guide but while my FoundationDB server looks like it comes up fine I cant get an app with a java client connected to it. Calls to DirectoryLayer.getDefault().createOrOpen time out and I’m still working through troubleshooting it.

5

Is you java app running in docker-compose as well, or are you trying to have a dockerized deployment of FDB that is used by a Java app running directly on your host machine? The latter scenario is trickier. The FDB_NETWORKING_MODE environment variable will let you control whether the database is accessible from the containers or from the host, but there’s no option today for both. One consequence of that is that if you want it to be accessible from the host, you can’t use docker-compose to spin up multiple instances of FDB, because they won’t be able to communicate with each other.

1 Like
6

I have both FDB and app running in docker-compose. I plan to simplify my docker-compose setup and throw it up on github in the next day or two but I dont think I’m doing anything differently compared to other samples I’ve seen. Essentially my FDB docker-compose setup is

fdb-coordinator:
  image: foundationdb/foundationdb:6.1.12
  enviornment:
    FDB_COORDINATOR: fdb-coordinator

App portion depends on fdb-coordinator and also sets the FDB_COORDINATOR env var. As part of the app Dockerfile I copy the create_cluster_file script from foundationdb/foundationdb:6.1.12

7

Sample docker-compose java app with foundationdb which also encounters time out exception https://github.com/seancarroll/fdb-docker-compose-sample

2 Likes
8

Maybe this is happening somewhere in your app and I’m missing it, but I believe you need to add an additional configuration step on database creation time though the CLI:

fdbcli -C "${cluster_file_path}" --exec "configure new single memory ; status"

That’s what this line here is doing in one of our sample applications for Docker: https://github.com/apple/foundationdb/blob/3bb0c12ce9c18ce7b334d6fed349d5e39f2c7d60/packaging/docker/samples/local/start.bash#L33

There’s some amount of trickiness here in that you only want to run that configuration step when you start up FDB for the first time. (It’s not included by default in the docker file because you don’t want to do it when, say, bouncing all instances of an existing FDB cluster.)

9

Yep thats what I discovered as well but forgot to add an update here :slight_smile:

1 Like