I’m wondering if there is prior art of how much resources are needed to build FoundationDB in the Docker container (using cmake). I’m not 100% sure, but it looks like the cc1plus process is getting OOM killed. Increasing the memory got the build to go further, but it would be useful if someone had a benchmark of what resources were need.
For reference, I’m trying to build on a Mac to run on Linux, hence Docker. Most recently, it died during Building CXX object fdbserver/CMakeFiles/fdbserver.dir/MasterProxyServer.actor.g.cpp.o if that is at all helpful.
Currently our memory requirements for building are pretty insane I think last time I measured it was around 8GB per core. So if you reduce parallelism (with -j in make and ninja) you should be able to compile (although slower).
There is this PR which will improve on this significantly: https://github.com/apple/foundationdb/pull/1904 Sadly this currently can’t be merged as it will break the Visual Studio based Windows build.
So as soon as the deprecated build system is completely removed we can hopefully improve the build experience significantly.
With 8GB, I’ve managed to get away with building with -j2 … most of the time? This situation is also made better if you compile with clang instead of gcc (the latter being the default).
FWIW, we internally run some distcc machines for this reason, so that developer experience isn’t horrifically painful for us, but that’s not an easy (or cheap?) thing to maintain as an external developer.
8 GB + -j2 with gcc isn’t working for me. What version of clang should I be using? It doesn’t seem like there is one installed in in the Docker container, and I haven’t found any documentation stating which versions of clang are supported.
The newest version of clang should work - I think clang 7 and newer works (although we don’t test them regularly, so it is hard to say without trying).
Our Docker image uses CentOS 6 for which there is no clang support. We probably should create a CentOS 7 container with clang to support clang for other people…
So you can take any Linux distribution and install clang and the other FDB dependencies manually. Sadly this requires some work, though FDB doesn’t have too many requirements.
Thanks. I’ll give that a try. I tried clang 3.4.2 (the CentOS 6 default) which built neither with libstdc++ nor libc++. The former choked on the tr1 hash map and the latter choked on cstdint not being present.
I’m happy to send out a PR adding that if you think it would be helpful. More broadly, I’m happy to set up a CentOS 7 Dockerfile for clang based development as long as it will be run on CI along with the gcc build.
Interesting, that should not be the case though? We build with clang and libc++ on MacOS, so I wonder what the problem here is…
On which system did you build this? Was this a CentOS 7 or something else?
That would be helpful, thanks! But this flag should not be added like that. Instead (in ConfigureCompiler) this line needs to be added at the right places:
I used Clang 7.0.0-3~ubuntu0.18.04.1. I’m not sure the libc++ version.
Comically enough, installing GCC was enough to break the Clang build as the linker could no longer find libstdc++. I have given up on building on my laptop and have a VM I can use for now.
I think last time I measured it was around 8GB per core. So if you reduce parallelism (with