Looking for community support for the Windows build

Development
21

Welcome to C++ I guess… To be fair you are comparing generating byte code (in .NET most optimizations are done at runtime) vs generating optimized machine code. But even then, something like go or C compile times are often orders of magnitude lower than C++ compile times.

I know, I just wanted to point out that with MSVC counting compiler instances is not a good way of measuring parallelism. But it sounds like this is something that works for MSVC but is now broken for clang-cl and as a result the only parallelism you’re getting is across projects which is not very useful because fdbserver is so much larger than anything else?

In that case I am not sure what to try next… Does this new thing help?

22

I think we could say it does help a little bit *

image
image953×919 103 KB

*: but only for the very large projects where there’s enough files to keep the cores busy. The smaller projects still take relatively long with the typical fan-out symptom where the last large file takes longer than the rest one a single core, but overall a huge improvement!

It’s funny because this page is the fist one I saw, but I discarded it because I was not aware of the difference of behavior with msbuild with multiple files, and was looking for something else.

Edit: haha, setting CL_MPCount=48 does push it to the limit!

image

Build “completed” in a bit over 5 minutes (Release build, after a full clean).

Time Elapsed 00:05:13.95

Now we’re talking!

23

This is awesome, I am super impressed by your work! It took me in the past way longer to get done much less :smiley:

~5 minutes build times is amazing. My machine has only 8 cores, so I guess it’s not an apples to apples comparison.

I am looking forward to a PR. Then it seems we should move to clang and life on Windows will be much better

24

So it seems that docker is indeed somewhat working. I created a Windows docker file (currently it doesn’t include clang and I didn’t yet manage to actually test whether this can be used to build FDB). Do you think this would be a fruitful road to go down?

One of the drawbacks of the Windows (and macOS) build is that it’s hard to make it reproducible. This seems to work somewhat. For example this will give me a powershell prompt within a docker container that has msbuild, cmake, etc installed and in the path (the -m is super important because default is 1GB):

docker run -it --rm -v D:\develop:C:\develop -m 30GB fdb

Getting Docker on Windows working properly took me quite long, especially because it took me comically long to figure out that by default docker tries to run everything as a linux container and turns out you can’t install Windows in the Linux subsystem :smiley:

25

Messing with C++ and cross-platforms build systems reminds me why I prefer staying off of it as much as possible ^^;

I’m trying to move all the hacks I’ve done to my home machine which is “untouched” from all this mess. I’ll try to re-add fixes one by one, because I had to do shotgun debugging and most of the changes I’ve made probably have no real impact: most of the underlying causes where mostly fixed by defining NOMINMAX, BOOST_USE_WINDOWS_H and WIN32_LEAN_AND_MEAN, plus one or two issues with variables named like restricted keywoard (like interface).

What’s funny (or not) is that it took me as long as you to figure it out that I could only run windows containers on windows hosts :slight_smile: Since dotnet is cross platform, we use linux images for our containers and things are much easier.

26

I created a pull request with the minimal set of changes required to build for release and produce a working MSI: Build on Windows using VS 2019 + LLVM/Clang by KrzysFR · Pull Request #4258 · apple/foundationdb · GitHub

There’s still some weirdness: when using OPEN_FOR_IDE, the produced files do not build with strange errors. Some errors are double declaration of some variables defines as ‘state int foo’, or using them from a catch block when they are defined inside a try block. These errors do not happen when building without OPEN_FOR_IDE??

27

Debug build is still broken. Fixed an issue with template duplication, but the last issue seems to be during linking with calloc:

       Link:
         C:\Program Files (x86)\Microsoft Visual Studio\2019\Preview\VC\Tools\Llvm\x64\bin\lld-link.exe /OUT:"C:\temp\fdb\bin\Debug\fdbserver.exe" /INCREMENTAL ..\lib\Debug\fdbclient.lib ..\lib
         \Debug\fdb_sqlite.lib ..\lib\Debug\fdbrpc.lib ..\lib\Debug\thirdparty.lib ..\lib\Debug\flow.lib winmm.lib psapi.lib kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib
          oleaut32.lib uuid.lib comdlg32.lib advapi32.lib /MANIFEST /MANIFESTUAC:"level='asInvoker' uiAccess='false'" /manifest:embed /DEBUG:FULL /PDB:"C:/temp/fdb/bin/Debug/fdbserver.pdb" /SUB
         SYSTEM:CONSOLE /DYNAMICBASE /NXCOMPAT /IMPLIB:"C:/temp/fdb/bin/Debug/fdbserver.lib"   /machine:x64 fdbserver.dir\Debug\ApplyMetadataMutation.obj
...
   100>lld-link : error : undefined symbol: __declspec(dllimport) calloc [C:\temp\fdb\fdbserver\fdbserver.vcxproj]
         >>> referenced by C:\DATA\Git\GitHub\foundationdb\fdbrpc\libcoroutine\Coro.c:80
         >>>               thirdparty.lib(Coro.obj):(Coro_new)
         >>> referenced by C:\DATA\Git\GitHub\foundationdb\fdbrpc\libcoroutine\Coro.c:109
         >>>               thirdparty.lib(Coro.obj):(Coro_allocStackIfNeeded)

Any idea why the behavior is different between Release and Debug build regarding memory allocation? Maybe debug builds use a different allocator to track leaks/bugs?

28

This is a new error, didn’t see this before. It is kind of weird that this wouldn’t link in debug even though it works in Release.

This being said, I believe nobody is really using debug mode (on any platform – debugging FDB in an actual debug is just not very practical because of how flow makes this hard), so I am not sure how much of a problem that is?

I am currently in the process of moving from Coro to coroutine2. It won’t go into 6.3, but maybe if this works we might just live with debug build not working on Windows?