I’m not sure how many people run FDB Windows build in prod clusters at scale.
Windows build is not used in the companies that use FDB in prod at scale, as far as I know.
Right now, windows build is only “guaranteed” to be compilable, but not tested in real clusters.
It seems unwise to use windows build for serious use cases.
I’d suggest we consider dropping the official support for Windows build and looking for community support for the Windows build.
What do people think?
Given that we don’t properly test on Windows, I think we should drop official windows support. I believe users would be better off using WSL - even if it is slightly slower, I would have way more confidence that it doesn’t break anything.
We’re still here! o/
Been running on Windows since forever without many issues.
Note that though the fdb server binaries install and run on WSL 2 there is an issue with the static IP: Every time the WSL2 vm restarts, it can change its local IP address (used between the windows host and linux guest) so the fdb.cluster file becomes invalid and need to be updated by hand every time…
If you drop support for the client, we - for example - would be severely impacted by that: we develop using Visual Studio or Visual Studio Code on Windows, meaning we could not easily test/debug or run our backend applications.
Dropping support for the server part for Windows, would be less an issue.
It is not uncommon to have database systems that require a specific OS on the server, but they usually have clients for all platforms/languages, but requiring a specific OS for the clients also would make it harder to start develop for it.
Thanks for your input, Christophe!
May I ask which FDB version you are currently running?
I think having a functioning windows client is a fair requirement. One avenue I still want to explore is using clang to compile FDB on Windows - this would at least give us some continuity and clang is officially supported by Microsoft (so you can link binaries produced by MSVC and clang).
The problem I have been running into was that the clang version that Microsoft provides was compiled to 32bit intel. But compilation would occasionally use more than 3GB of main memory for some files.
A mix of 6.0, 6.1 and 6.2, waiting for 6.3 to be stable to upgrade, but most of these are already on linux or could be linux.
We even have much older versions than are running fine and that did not need any of the new 6.x features or caused problems so were left alone. All of these are “small scale” though.
But on the client side, most of the development, CI and testing is happening on Windows (either talking to a local fdb server for unit testing, or a remote linux cluster). The test servers have multiple versions installed in // (so that we can test older versions that were targeting 5.x and 6.0/6.1).
Have you tried the latest Visual Studio 2019 Preview? I see references to 64-bit clang starting from 16.1 preview 1 (which is already old from mid-2019) here: Clang/LLVM Support in Visual Studio - C++ Team Blog
In the latest VS 2019 version on my machine (non preview) I also see them, after installing the components manually:
This document mentions seems to imply there a x64 binaries for all the llvm/clang tools : Clang/LLVM support in Visual Studio projects | Microsoft Learn
By default, clang-cl.exe is located in %VCINSTALLDIR%\Tools\Llvm\bin\ and %VCINSTALLDIR%\Tools\Llvm\x64\bin
Here is what I see in this folder on my machine:
The non-x64 folder has the sames binaries and they are all a bit smaller, which would be expected if they were built for 32-bits.
I found older pages talking about hacking the 32-bit versions with editbin to make them Large Address Aware (see Developer Community ) but it seems that this is not required anymore?
I did not try this yet – and currently I don’t have a machine to try (I used my personal gaming PC back then).
Do you think you could try whether this works? The cmake part to configure the compiler on Windows is relatively short…
I followed the CMAKE instructions but I’m having an issue with Ninja not recognizing the VS 2019 C# compiler. Even hacking the EnableCsharp file to force mono detection, makes it fail later on (does not find a valid linker for projects written in C#)
I had to switch to a different generator by running cmake -G "Visual Studio 16 2019" -A x64 -DCMAKE_BUILD_TYPE=Release -DUSE_WERROR=ON ..
CMAKE runs until the end, and the builds fails with compiler is out of heap space like you probably encountered.
I had an issue building xxhash.h: it was trying to include <stdalign.h> instead of declaring the macro for MSVC: I manually commented everything except line 640 to work around that.
I’m not sure what I must now do to switch to LLVM/clang to test things out?
Here are the steps I used:
- Installed CMAKE 3.19.3 from https://cmake.org/download/
- Installed Mono 6.12.0 from https://www.mono-project.com/download/stable/, added to PATH
- Downloaded Ninja 1.10.2 from https://github.com/ninja-build/ninja/releases
- note: was lazy and copied ninja.exe to System32, you should follow these instructions instead => https://github.com/rwols/CMakeBuilder/wiki/Ninja-for-Windows-Installation-Instructions
- Installed Open JDK 15, added to PATH and JAVA_HOME
Using VS 2019 Developper Prompt (imporant!)
- cd into fdb repo
- mkdir distrib
- cd distrib
- cmake -G "Visual Studio 16 2019" -A x64 -DCMAKE_BUILD_TYPE=Release -DUSE_WERROR=ON ..
- msbuild /p:Configuration=Release /p:PreferredToolArchitecture=x64 foundationdb.sln
> fails with 'compiler is out of heap space' when building flow
Is there documentation that claims building with Ninja on Windows is supported? If so we need to correct this. I checked the readme and it says that on Windows you should use Visual Studio to build.
I think setting USE_WERROR to ON is not a good idea as the Windows builds generate a lot of warnings (this is a known issue and so far nobody took the time to fix those). I think for the target you would need to set Visual Studio 16 2019 Win64 (not sure whether the -A argument does the same thing.
I think in order to use clang you might need to pass the toolset to cmake (I only found this stackoverflow question – I don’t remember what I tried in the past).
The XXHash problem you run into is weird. Does VS16 support C11? If not the above macros should be correct. Sorry I don’t think anyone tested with VS16 yet…
The documentation first starts with a steps that do not specify a specific environment (https://github.com/apple/foundationdb/blob/master/README.md#compiling-from-source) so I started following it, and only found out about the Windows specific with an entire different set of step after scrolling down to the end of the page…
Followed the steps (they are a bit outdated version-wise) and had to install a few additional things. Most important was llvm-utils to make it work with cmake and VS2019: GitHub - zufuliu/llvm-utils: LLVM/Clang toolsets for Visual Studio 2022, 2019, 2017, 2015, 2013, 2012 and 2010.
EDIT : DISREGARD! This toolkit is too old, and does not work properly anymore! I swtiched to used VS ‘ClangCL’ toolset and it works better, see next message!
After downloading the LLVM_VS2017.zip from Release v20.03 · zufuliu/llvm-utils · GitHub and run install.bat, it copied the correct files for both vs2017 and vs2019. I just had to add -T "LLVM_v142" to CMAKE and it generated the proper project files.
Though it looks like it does not enable parallel build, because the build tooki FOREVER and only used a single core. It took 1h10min and build most of the binaries though there are a few errors remaining.
While it was running I was able to confirm that the cl-lang.exe process is indeed running as x64.
"C:\Data\Git\GitHub\foundationdb\distrib\foundationdb.sln" (default target) (1) ->
"C:\Data\Git\GitHub\foundationdb\distrib\ALL_BUILD.vcxproj.metaproj" (default target) (2) ->
"C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.vcxproj.metaproj" (default target) (76) ->
"C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.vcxproj" (default target) (81) ->
(ClCompile target) ->
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(
898,48): error : conflicting types for 'GetProcessTimes' [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.v
cxproj]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(
914,49): error : functions that differ only in their return type cannot be overloaded [C:\Data\Git\GitHub\foundationdb\
distrib\fdbbackup\fdbbackup.vcxproj]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(
915,50): error : conflicting types for 'FindFirstFileA' [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.vc
xproj]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(
916,50): error : conflicting types for 'FindNextFileA' [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.vcx
proj]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(
920,51): error : conflicting types for 'CreateMutexA' [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.vcxp
roj]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(
924,48): error : conflicting types for 'UnmapViewOfFile' [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.v
cxproj]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(
925,51): error : conflicting types for 'CreateSemaphoreA' [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.
vcxproj]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(
928,51): error : conflicting types for 'CreateFileMappingA' [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbacku
p.vcxproj]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(
931,51): error : conflicting types for 'CreateFileA' [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.vcxpr
oj]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(
932,49): error : conflicting types for 'GetSystemInfo' [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.vcx
proj]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(
933,48): error : conflicting types for 'FlushViewOfFile' [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.v
cxproj]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(
937,48): error : conflicting types for 'GetFileSizeEx' [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.vcx
proj]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(
944,48): error : conflicting types for 'CreateDirectoryA' [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.
vcxproj]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(
946,48): error : functions that differ only in their return type cannot be overloaded [C:\Data\Git\GitHub\foundationdb\
distrib\fdbbackup\fdbbackup.vcxproj]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(
950,48): error : conflicting types for 'SetFilePointerEx' [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.
vcxproj]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(
953,48): error : conflicting types for 'LockFileEx' [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.vcxpro
j]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(
954,48): error : conflicting types for 'UnlockFileEx' [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.vcxp
roj]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(
955,48): error : conflicting types for 'WriteFile' [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.vcxproj
]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(
956,48): error : conflicting types for 'ReadFile' [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.vcxproj]
"C:\Data\Git\GitHub\foundationdb\distrib\foundationdb.sln" (default target) (1) ->
"C:\Data\Git\GitHub\foundationdb\distrib\ALL_BUILD.vcxproj.metaproj" (default target) (2) ->
"C:\Data\Git\GitHub\foundationdb\distrib\fdbservice\fdbmonitor.vcxproj.metaproj" (default target) (100) ->
"C:\Data\Git\GitHub\foundationdb\distrib\fdbservice\fdbmonitor.vcxproj" (default target) (101) ->
C:\Data\Git\GitHub\foundationdb\fdbservice\FDBService.cpp(414,54): error : cannot pass object of non-trivial type 'st
d::string' (aka 'basic_string<char, char_traits<char>, allocator<char>>') through variadic function; call will abort at
runtime [-Wnon-pod-varargs] [C:\Data\Git\GitHub\foundationdb\distrib\fdbservice\fdbmonitor.vcxproj]
C:\Data\Git\GitHub\foundationdb\fdbservice\FDBService.cpp(418,54): error : cannot pass object of non-trivial type 'st
d::string' (aka 'basic_string<char, char_traits<char>, allocator<char>>') through variadic function; call will abort at
runtime [-Wnon-pod-varargs] [C:\Data\Git\GitHub\foundationdb\distrib\fdbservice\fdbmonitor.vcxproj]
C:\Data\Git\GitHub\foundationdb\fdbservice\FDBService.cpp(841,5): error : first parameter of 'main' (argument count)
must be of type 'int' [C:\Data\Git\GitHub\foundationdb\distrib\fdbservice\fdbmonitor.vcxproj]
"C:\Data\Git\GitHub\foundationdb\distrib\foundationdb.sln" (default target) (1) ->
"C:\Data\Git\GitHub\foundationdb\distrib\ALL_BUILD.vcxproj.metaproj" (default target) (2) ->
"C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj.metaproj" (default target) (102) ->
"C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj" (default target) (107) ->
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\fileapi.h(59,1): error : conflicting types for 'Create
DirectoryA' [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\fileapi.h(87,1): error : conflicting types for 'Create
FileA' [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\fileapi.h(233,1): error : conflicting types for 'FindF
irstFileA' [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\fileapi.h(322,1): error : conflicting types for 'FindN
extFileA' [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\fileapi.h(651,1): error : functions that differ only i
n their return type cannot be overloaded [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\fileapi.h(681,1): error : conflicting types for 'GetFi
leSizeEx' [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\fileapi.h(690,1): error : functions that differ only i
n their return type cannot be overloaded [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\fileapi.h(984,1): error : conflicting types for 'LockF
ileEx' [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\fileapi.h(1024,1): error : conflicting types for 'Read
File' [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\fileapi.h(1150,1): error : conflicting types for 'SetF
ilePointerEx' [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\fileapi.h(1209,1): error : conflicting types for 'Unlo
ckFileEx' [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\fileapi.h(1221,1): error : conflicting types for 'Writ
eFile' [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\fileapi.h(1462,1): error : functions that differ only
in their return type cannot be overloaded [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\synchapi.h(428,1): error : conflicting types for 'Crea
teMutexA' [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\processthreadsapi.h(106,1): error : conflicting types
for 'GetProcessTimes' [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\sysinfoapi.h(95,1): error : conflicting types for 'Get
SystemInfo' [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\memoryapi.h(273,1): error : conflicting types for 'Flu
shViewOfFile' [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\memoryapi.h(282,1): error : conflicting types for 'Unm
apViewOfFile' [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\libloaderapi.h(186,1): error : conflicting types for '
FreeLibrary' [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
"C:\Data\Git\GitHub\foundationdb\distrib\foundationdb.sln" (default target) (1) ->
"C:\Data\Git\GitHub\foundationdb\distrib\documentation\buildsphinx.vcxproj.metaproj" (default target) (126) ->
"C:\Data\Git\GitHub\foundationdb\distrib\documentation\buildsphinx.vcxproj" (default target) (127) ->
(CustomBuild target) ->
C:\Program Files (x86)\Microsoft Visual Studio\2019\Preview\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(240
,5): error MSB8066: Custom build for 'C:\Data\Git\GitHub\foundationdb\distrib\CMakeFiles\0a42c1b5e737061c329a291dcab471
e1\venv_setup.rule;C:\Data\Git\GitHub\foundationdb\distrib\CMakeFiles\2d165168042997a2d819c3db7ea1bfb5\buildsphinx.rule
' exited with code 9009. [C:\Data\Git\GitHub\foundationdb\distrib\documentation\buildsphinx.vcxproj]
2 Warning(s)
42 Error(s)
OK, so llvm-utils was not the correct choice, it seems unmaintained and was the only solution before official support of clang in recent VS versions. I changed the toolset to ClangCL which is the name of the official Microsoft Platform Toolset for clang/llvm.
cmake -G "Visual Studio 16 2019" -A x64 -T"ClangCL" -DCMAKE_BUILD_TYPE=Release ..
msbuild /p:Configuration=Release /p:PreferredToolArchitecture=x64 foundationdb.sln
This makes so that all vcproj have this toolset selected:
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'" Label="Configuration">
<ConfigurationType>Utility</ConfigurationType>
<CharacterSet>MultiByte</CharacterSet>
<PlatformToolset>ClangCL</PlatformToolset>
</PropertyGroup>
And VS seems to like this better and even selected ISO C++ 17!
This fixed the build issues like xxhash for example (macro correctly identifies MSVC).
The only problem is that the build is still single threaded, and probably take another hour to complete… I’ll keep you posted with the result.
I’m pretty sure there are flags or parameters that must be passed to clang to build in // (I would be very surprised if clang would take 1 hour on linux for example!). Any ideas?
Thank you so much for looking into this!!! Also a more general question: I think (but here my Windows knowledge might not be good enough and I miss something that makes this not a good idea) we could create a docker image for Windows so that the build is easily reproducible by anyone.
That shouldn’t be. Did you try using the msbuild command from the readme?
Tried both msbuild and from VS itself, they do the same thing.
The build failed after 20 minutes with the same errors as above, but some of the binaries where build:
Errors:
"C:\Data\Git\GitHub\foundationdb\distrib\foundationdb.sln" (default target) (1) ->
"C:\Data\Git\GitHub\foundationdb\distrib\ALL_BUILD.vcxproj.metaproj" (default target) (2) ->
"C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.vcxproj.metaproj" (default target) (76) ->
"C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.vcxproj" (default target) (81) ->
(ClCompile target) ->
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(898,48): error : conflicting types for 'Get
ProcessTimes' [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.vcxproj]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(914,49): error : functions that differ only
in their return type cannot be overloaded [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.vcxproj]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(915,50): error : conflicting types for 'Fin
dFirstFileA' [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.vcxproj]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(916,50): error : conflicting types for 'Fin
dNextFileA' [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.vcxproj]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(920,51): error : conflicting types for 'Cre
ateMutexA' [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.vcxproj]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(924,48): error : conflicting types for 'Unm
apViewOfFile' [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.vcxproj]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(925,51): error : conflicting types for 'Cre
ateSemaphoreA' [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.vcxproj]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(928,51): error : conflicting types for 'Cre
ateFileMappingA' [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.vcxproj]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(931,51): error : conflicting types for 'Cre
ateFileA' [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.vcxproj]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(932,49): error : conflicting types for 'Get
SystemInfo' [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.vcxproj]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(933,48): error : conflicting types for 'Flu
shViewOfFile' [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.vcxproj]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(937,48): error : conflicting types for 'Get
FileSizeEx' [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.vcxproj]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(944,48): error : conflicting types for 'Cre
ateDirectoryA' [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.vcxproj]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(946,48): error : functions that differ only
in their return type cannot be overloaded [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.vcxproj]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(950,48): error : conflicting types for 'Set
FilePointerEx' [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.vcxproj]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(953,48): error : conflicting types for 'Loc
kFileEx' [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.vcxproj]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(954,48): error : conflicting types for 'Unl
ockFileEx' [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.vcxproj]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(955,48): error : conflicting types for 'Wri
teFile' [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.vcxproj]
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/interprocess/detail/win32_api.hpp(956,48): error : conflicting types for 'Rea
dFile' [C:\Data\Git\GitHub\foundationdb\distrib\fdbbackup\fdbbackup.vcxproj]
"C:\Data\Git\GitHub\foundationdb\distrib\foundationdb.sln" (default target) (1) ->
"C:\Data\Git\GitHub\foundationdb\distrib\ALL_BUILD.vcxproj.metaproj" (default target) (2) ->
"C:\Data\Git\GitHub\foundationdb\distrib\fdbservice\fdbmonitor.vcxproj.metaproj" (default target) (100) ->
"C:\Data\Git\GitHub\foundationdb\distrib\fdbservice\fdbmonitor.vcxproj" (default target) (101) ->
C:\Data\Git\GitHub\foundationdb\fdbservice\FDBService.cpp(414,54): error : cannot pass object of non-trivial type 'std::string' (aka 'basic_string<char, char_tr
aits<char>, allocator<char>>') through variadic function; call will abort at runtime [-Wnon-pod-varargs] [C:\Data\Git\GitHub\foundationdb\distrib\fdbservice\fdbmo
nitor.vcxproj]
C:\Data\Git\GitHub\foundationdb\fdbservice\FDBService.cpp(418,54): error : cannot pass object of non-trivial type 'std::string' (aka 'basic_string<char, char_tr
aits<char>, allocator<char>>') through variadic function; call will abort at runtime [-Wnon-pod-varargs] [C:\Data\Git\GitHub\foundationdb\distrib\fdbservice\fdbmo
nitor.vcxproj]
C:\Data\Git\GitHub\foundationdb\fdbservice\FDBService.cpp(841,5): error : first parameter of 'main' (argument count) must be of type 'int' [C:\Data\Git\GitHub\f
oundationdb\distrib\fdbservice\fdbmonitor.vcxproj]
"C:\Data\Git\GitHub\foundationdb\distrib\foundationdb.sln" (default target) (1) ->
"C:\Data\Git\GitHub\foundationdb\distrib\ALL_BUILD.vcxproj.metaproj" (default target) (2) ->
"C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj.metaproj" (default target) (102) ->
"C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj" (default target) (107) ->
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\fileapi.h(59,1): error : conflicting types for 'CreateDirectoryA' [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\fileapi.h(87,1): error : conflicting types for 'CreateFileA' [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\fileapi.h(233,1): error : conflicting types for 'FindFirstFileA' [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\fileapi.h(322,1): error : conflicting types for 'FindNextFileA' [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\fileapi.h(651,1): error : functions that differ only in their return type cannot be overloaded [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\fileapi.h(681,1): error : conflicting types for 'GetFileSizeEx' [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\fileapi.h(690,1): error : functions that differ only in their return type cannot be overloaded [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\fileapi.h(984,1): error : conflicting types for 'LockFileEx' [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\fileapi.h(1024,1): error : conflicting types for 'ReadFile' [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\fileapi.h(1150,1): error : conflicting types for 'SetFilePointerEx' [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\fileapi.h(1209,1): error : conflicting types for 'UnlockFileEx' [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\fileapi.h(1221,1): error : conflicting types for 'WriteFile' [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\fileapi.h(1462,1): error : functions that differ only in their return type cannot be overloaded [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\synchapi.h(428,1): error : conflicting types for 'CreateMutexA' [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\processthreadsapi.h(106,1): error : conflicting types for 'GetProcessTimes' [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\sysinfoapi.h(95,1): error : conflicting types for 'GetSystemInfo' [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\memoryapi.h(273,1): error : conflicting types for 'FlushViewOfFile' [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\memoryapi.h(282,1): error : conflicting types for 'UnmapViewOfFile' [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um\libloaderapi.h(186,1): error : conflicting types for 'FreeLibrary' [C:\Data\Git\GitHub\foundationdb\distrib\fdbserver\fdbserver.vcxproj]
"C:\Data\Git\GitHub\foundationdb\distrib\foundationdb.sln" (default target) (1) ->
"C:\Data\Git\GitHub\foundationdb\distrib\documentation\buildsphinx.vcxproj.metaproj" (default target) (126) ->
"C:\Data\Git\GitHub\foundationdb\distrib\documentation\buildsphinx.vcxproj" (default target) (127) ->
(CustomBuild target) ->
C:\Program Files (x86)\Microsoft Visual Studio\2019\Preview\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(240,5): error MSB8066: Custom build for 'C:\Da
ta\Git\GitHub\foundationdb\distrib\CMakeFiles\0a42c1b5e737061c329a291dcab471e1\venv_setup.rule;C:\Data\Git\GitHub\foundationdb\distrib\CMakeFiles\2d165168042997a2
d819c3db7ea1bfb5\buildsphinx.rule' exited with code 9009. [C:\Data\Git\GitHub\foundationdb\distrib\documentation\buildsphinx.vcxproj]
2 Warning(s)
42 Error(s)
Time Elapsed 00:20:42.39
note: I’m using the 6.3 branch to build
Making a bit of progress, there are a lot of compilation errors due to slight differences in expected types (int vs uint16_t, size_t vs int, const_cast issues, …). One of the bigest issue is defining so that it uses the winapi.h definitions to get rid of duplicate definition of the win api file apis.
There are also some issues with building the documentation (html, some sphinx comment not found, …) that may be because of missing dependencies.
The only remaining issue I have are the following:
C:\Data\Git\GitHub\foundationdb\flow/Arena.h(181,59): error : too few arguments provided to function-like macro invocation [C:\Data\Git\GitHub\foundationdb\distrib\flow\flow.vcxproj]
C:\Data\Git\GitHub\foundationdb\flow/Arena.h(181,27): error : comparison between pointer and integer ('size_t' (aka 'unsigned long long') and 'int (*)() noexcept') [C:\Data\Git\GitHub\foundationdb\distrib\flow\flow.vcxproj]
C:\Data\Git\GitHub\foundationdb\flow/Arena.h(186,59): error : too few arguments provided to function-like macro invocation [C:\Data\Git\GitHub\foundationdb\distrib\flow\flow.vcxproj]
C:\Data\Git\GitHub\foundationdb\flow/Arena.h(186,27): error : comparison between pointer and integer ('size_t' (aka 'unsigned long long') and 'int (*)() noexcept') [C:\Data\Git\GitHub\foundationdb\distrib\flow\flow.vcxproj]
C:\Data\Git\GitHub\foundationdb\flow/Arena.h(407,70): error : too few arguments provided to function-like macro invocation [C:\Data\Git\GitHub\foundationdb\distrib\flow\flow.vcxproj]
C:\Data\Git\GitHub\foundationdb\flow/Arena.h(418,48): error : too few arguments provided to function-like macro invocation [C:\Data\Git\GitHub\foundationdb\distrib\flow\flow.vcxproj]
C:\Data\Git\GitHub\foundationdb\flow/Arena.h(407,37): error : comparison between pointer and integer ('std::basic_string<char, std::char_traits<char>, std::allocator<char>>::size_type' (aka 'unsigned long long') and 'int (*)() noexcept') [C:\Data\Git\GitHub\foundationdb\distrib\flow\flow.vcxproj]
C:\Data\Git\GitHub\foundationdb\flow/Arena.h(418,16): error : comparison between pointer and integer ('std::basic_string<char, std::char_traits<char>, std::allocator<char>>::size_type' (aka 'unsigned long long') and 'int (*)() noexcept') [C:\Data\Git\GitHub\foundationdb\distrib\flow\flow.vcxproj]
C:\Data\Git\GitHub\foundationdb\flow/Arena.h(513,25): error : expected unqualified-id [C:\Data\Git\GitHub\foundationdb\distrib\flow\flow.vcxproj]
C:\Data\Git\GitHub\foundationdb\flow/Arena.h(685,11): error : expected unqualified-id [C:\Data\Git\GitHub\foundationdb\distrib\flow\flow.vcxproj]
C:\Data\Git\GitHub\foundationdb\flow/Arena.h(686,49): error : expected unqualified-id [C:\Data\Git\GitHub\foundationdb\distrib\flow\flow.vcxproj]
C:\Data\Git\GitHub\foundationdb\flow/Arena.h(692,11): error : expected unqualified-id [C:\Data\Git\GitHub\foundationdb\distrib\flow\flow.vcxproj]
C:\Data\Git\GitHub\foundationdb\flow/Arena.h(693,49): error : expected unqualified-id [C:\Data\Git\GitHub\foundationdb\distrib\flow\flow.vcxproj]
C:\Data\Git\GitHub\foundationdb\flow/flat_buffers.h(339,14): error : expected unqualified-id [C:\Data\Git\GitHub\foundationdb\distrib\flow\flow.vcxproj]
C:\Data\Git\GitHub\foundationdb\flow/flat_buffers.h(339,14): error : expected ';' after expression [C:\Data\Git\GitHub\foundationdb\distrib\flow\flow.vcxproj]
C:\Data\Git\GitHub\foundationdb\flow/flat_buffers.h(339,14): error : expected expression [C:\Data\Git\GitHub\foundationdb\distrib\flow\flow.vcxproj]
C:\Data\Git\GitHub\foundationdb\flow/flat_buffers.h(339,14): error : expected ';' after expression [C:\Data\Git\GitHub\foundationdb\distrib\flow\flow.vcxproj]
C:\Data\Git\GitHub\foundationdb\flow/flat_buffers.h(339,14): error : expected expression [C:\Data\Git\GitHub\foundationdb\distrib\flow\flow.vcxproj]
C:\Data\Git\GitHub\foundationdb\flow/flat_buffers.h(339,14): error : expected ':' [C:\Data\Git\GitHub\foundationdb\distrib\flow\flow.vcxproj]
One such example is the comparison between a size_t and int in the assert: size_t is a long long, but the assert uses std::numeric_limits<int> ? (multiple instances of the same issue, comparing a size/length that is an int, with a size_t)
Almost all instances of std::max(..) or std::min(..) fail with the same weird macro expansion issue (expected unqualified-id) and I don’t understand what this message means.
They all seem to point to a max() and min() macros somewhere in C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\shared\minwindef.h that has nothing to do with it. As if it did not see the std:: prefix and did not understand it was a different method?
#ifndef NOMINMAX
#ifndef max
#define max(a,b) (((a) > (b)) ? (a) : (b))
#endif
#ifndef min
#define min(a,b) (((a) < (b)) ? (a) : (b))
#endif
#endif /* NOMINMAX */
Plus, the corresponding min/max macros in minwindef.h are not defined because NOMINMAX is not defined so I really don’t understand why the callstack points to these ??
oh wow, you’re amazing! This progress is super cool to see! Can you make a draft PR so I can review your changes while they’re happening?
Regarding the documentation: I think we could just disable building the documentation on Windows, at least for now. On Linux we can build it just well.
For the min max thingy. I remember running into this multiple time and it’s always super annoying to debug (I think last time I had to look at preprocessed files to find the problem). I would suggest to first delete all occurrences of #define NOMINMAX (there are four I believe) and then add this line to cmake/ConfigureCompiler.cmake into the if(WIN32) block (somewhere around line 83):
add_compile_definitions(NOMINMAX)
This should then define it for all compilation units (I hope this helps).
This did the trick! 
Now onto some winsock issues!
(ClCompile target) ->
C:\Data\Git\GitHub\foundationdb\distrib\boostProject-prefix\src\boostProject\boost/asio/detail/socket_types.hpp(24,4): error :
WinSock.h has already been included [C:\Data\Git\GitHub\foundationdb\distrib\flow\flow.vcxproj]
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\shared\ws2def.h(238,16): error :
redefinition of 'sockaddr' [C:\Data\Git\GitHub\foundationdb\distrib\flow\flow.vcxproj]
Was able to build fdbserver !
C:\Data\Git\GitHub\foundationdb\distrib\bin\Release>fdbcli --version
FoundationDB CLI 6.3 (v6.3.11)
source version 5bd7e2ab6da6caf197e83a80651dcd913c65b2c4
protocol fdb00b063010001
C:\Data\Git\GitHub\foundationdb\distrib\bin\Release>fdbserver --version
FoundationDB 6.3 (v6.3.11)
source version 5bd7e2ab6da6caf197e83a80651dcd913c65b2c4
protocol fdb00b063010001
The java binding seems to have been built as well.
The html doc / sphinx is the last thing that fails to build.
Once I have a bit more time, I’ll try cleanup everything into a PR, because I spent most time randomly prodding the code with a stick until it compiles and it’s a bit ugly.
The last show stopper issue for me is that it is SLOOOW. Takes something like 30minutes just to build fdbserver …? Is this expected to be that slow even on linux ?
Do you know if there are, somewhere, some options that would force single-core compilation? I tried all the solutions I found to ask clang to build in //, but nothing has any effect. I see a single clang-cl.exe working on each project, locked to a single core.
–
Can run a server node from the CLI:
C:\Data\Git\GitHub\foundationdb\distrib\bin\Release>fdbserver.exe --cluster_file="R:\fdb\fdb.cluster" --datadir="R:\fdb\Data\9501" --listen_address="public" --logdir="R:\fdb\logs" --public_address="auto:9501"
And connect to it using the compiled fdbcli:
fdb> configure new single ssd
Database created
fdb> status details
Using cluster file `R:\fdb\fdb.cluster'.
Configuration:
Redundancy mode - single
Storage engine - ssd-2
Coordinators - 1
Usable Regions - 1
Cluster:
FoundationDB processes - 1
Zones - 1
Machines - 1
Memory availability - 8.0 GB per process on machine with least available
Fault Tolerance - 0 machines
Server time - 01/29/21 14:40:41
Data:
Replication health - (Re)initializing automatic data distribution
Moving data - unknown (initializing)
Sum of key-value sizes - unknown
Disk space used - 210 MB
Operating space:
Storage server - 3.1 GB free on most full server
Log server - 3.1 GB free on most full server
Workload:
Read rate - 0 Hz
Write rate - 0 Hz
Transactions started - 0 Hz
Transactions committed - 0 Hz
Conflict rate - 0 Hz
Backup and DR:
Running backups - 0
Running DRs - 0
Process performance details:
127.0.0.1:9501 ( 1% cpu; 1% machine; 0.000 Gbps; 0% disk IO; 0.1 GB / 8.0 GB RAM )
Coordination servers:
127.0.0.1:9501 (reachable)
Client time: 01/29/21 14:40:41
fdb>
``This is great news! Now we just need to find a way of replicating this on CI (I might look into docker if I have some time).
Yes building FDB is super slow. 30 minutes is on the long end, but it doesn’t sound crazy. This is C++ and templates…
Do you mean multi-core compilation? For msbuild something like /m:8 used to work for me. But this is incredibly volatile on Windows 
First question: did you check how much CPU this one clang-cl.exe this process uses? I am not sure about clang-cl but the way parallelism works with the visual studio compiler is different from what gcc on linux does. You see, on Linux if we want to compile 3 .cpp files we (aka make/ninja) would call 3 instances of our compiler in parallel. But this is not what Visual Studio does. Instead it will pass all three files at once to the compiler and it will compile the files together. So on Linux you would see 3 gcc processes each consuming ~100% CPU (assuming you have more than 3 cores). On Windows you would see one compiler running but consuming ~300% CPU. In order to figure this out I once spent 2 full days debugging a cmake issue on Windows.
One way I have seen this break in the past (where I spent 2 days finding a fix) has been if there are files (or even one file) within a project that is compiled with different command line arguments than the rest, above form of parallelism breaks. I have been very careful with code reviews since then to make sure we don’t do this anymore (this is why we create some weird mini-libraries in flow that we then statically link). But if you indeed don’t see parallelism I would suggest to check the .vsproj files to see whether some compilation units have different options passed to them.
It seems that this is currently also broken in our CI. I would suggest you don’t waste too much time on this, there might be something fundamentally broken
I work on a .NET code base that is at least an order of magnitude larger than fdb’s code base, and a full rebuild takes maximum one minute, and a partial build usually takes 10-15 seconds!
Most CI builds take 1 or 2 minutes on a slower VM, including the nuget restore, clean and full rebuild. Usually, it is the “npm ci” steps that takes more than half the build time!
I’m not talking about being able to build multiple separate projects in // (like msbuild or ninja can do), but the time it takes to build a single project. (so for example rebuilding a single project after making a one line change somewhere).
When building everything, at first I see multiple clang-cl crunching each on a few utility projects that depend on nothing, but once all the easy projects are done, there is the long chain of flow > fdb_rpc > fdbclient > fdbserver etc… where they each wait for the previous one to finish. Here there’s only one clang-cl working at a time and single-threaded.
When I was using MSVC as the compiler before switching to clang it would spend at most a dozen seconds per project, and ALL my 48 threads on my AMD ThreadRipper 3960X would be pegged at 100% for most of the duration of the build (that’s a LOT of threads crunching C/C++ code!). Granted it would start failing when it reached flow or fdbserver, but it would reach them quickly.
When I switched to clang-cl, only a single thread would be pegged at 100%, for dozens of minutes. On first approximation this would explain why the compile time appears ~48 times slower on my machine.
I tried using the /p:CL_MPCount=3 /m arguments on msbuild that is supposed to tell clang to use more threads, but I didn’t see any effect at all.