I’m working on release builds, artifacts, and packages (for the core foundationdb KV store), and wondering what people/organizations out there may be using the rpm or deb packages, or tried to use them. Are they useful, do they work?
At Apple we use the stand-alone binaries published as part of the tagged releases, and also the docker container images which are published to docker-hub and internal mirrors. We recently reviewed+merged a pull request with some fixes for the “versioned” rpm packages, which enable installing multiple versions (e.g. of client tools and library) in parallel on a system. But we have never published that “versioned” rpm package that our build system can generate, we only publish the regular rpm packages which can only have 1 version installed at once. And again, we don’t use those either. Should also be publishing the “versioned” rpm packages?
I’ve also been wrestling with the rpm debuginfo packages. We are trying to fix/enhance some internal usages of foundationdb client library and server to get debug symbols to more places where we want them for profiling etc. Currently the build system does not compress debuginfo sections for binaries/libraries during release builds, because the RPM packaging historically had issue with it. (The rpm packaging uses “debugedit” to edit source file paths in the debuginfo sections, the version we have now in our rockylinux9 build image does support compressed debuginfo.) The debuginfo compression makes a huge difference (300MB down to 50MB for libfdb_c.so, ~1GB reduction for fdbserver) and we want it for those stand-alone binaries and such. The problem is that building RPM packages is very slow, and even slower with compressed debuginfo, slowing down the release build noticeably. It’s also quite annoying to iterate on changes to this packaging. The debuginfo packages also contain the sources which are typically split out again into “debugsource” packages in modern rhel/fedora. I’m considering trying to skip the source files and skip the debugedit stuff, but it’s not easy (would need to re-implement debuginfo package instead of using cpack functionality for it).
So … who uses, or has tried to use, these packages? rpm, deb, tgz? debuginfo packages? stand-alone debug binaries? Would there be demand for official builds of the “versioned” rpm packages? Do some people build their own packages using the foundationdb repo? Due to how fdb cluster version upgrades are handled, I suspect that in many cases, the existing packages are not really useful …
Firstly, thanks for asking this question Pierce, much appreciated. It is a crucial question to ask frankly at least for a healthy community.
To answer briefly - yes, they’re useful and I’m using them.
I’d like add that if you were to build official Debian packages you’d benefit from Debian’s legendary security services. In addition, you’d have your software tested and used by a very large community of technical users, which includes large companies, some larger than Apple. I realize that building package for Debian (which would automatically get picked up by Ubuntu) is a bit more time consuming and would have to go through some hurdles, but there are lots of benefits. I would be willing to help and I think you’d find others equally willing.
(I’m really not sure why that comment was “temporarily” flagged, maybe it changed … I did read it as it was first posted though.)
It sounds like you would like there to be FoundationDB packages in the official Debian package repositories. That’s quite unlikely for a few reasons:
Debian packages can be uploaded to the official repos only by Debian Developers, or new ones in training being “sponsored” by an existing Debian Developer. It is very rare (if ever) that a maintainer of some open-source software can push their software into Debian official repositories somehow, without being a Debian Developer, or sponsored.
FoundationDB really wants to vendor a few dependencies at specific versions. This does not fit into the old-school linux distro model, where the software packages shall use other software packages as dependencies, to avoid duplication and enable security updates for a dependency to be managed in one place (the separate package).
FoundationDB makes the most sense in large-scale deployments, and when you’re doing large-scale deployments of something like this you really want to control the version more directly. Also, a default startup script or systemd unit is not very practical.
I love Debian, I’ve run it on my own fileserver and desktop workstation at home for many years, and I’ve used it for many container base images … but the stability of the official package repo is really best for all the stuff besides the “main app” you are running. So for example, on my server, of course it uses debian official packages for linux kernel, glibc, coreutils, systemd, etc, and also nginx. I don’t care about the latest nginx features, the version from 3 years ago was and is great for my purposes. But I install tailscale from their third-party apt repository, and transmission-bt from sources, because I want to control the versions of these more.
So, I don’t think it makes sense to even try to get FoundationDB packages available from official Debian repos. But a third-party apt package repo, like many other projects provide (like Docker, Tailscale, PosgreSQL, MariaDB, etc) would make sense. Unfortunately, at the moment, Apple does not have the motivation to maintain such a third-party apt package repo, we do not even really test the deb packages. Currently, it is up to other users of deb packages in the community to report problems, and it would also be up to them to try to propose enhancements.
So, do let us know how the packages are actually working for various use cases
Thanks for the detailed reply. You’re right about Debian’s policies around uploading code to official repos. Though I’m not a DD I’ve worked with Debian and know DDs who’ve sponsored packages for me. If there’s a need I’m happy to help here with packaging. (It’s a volunteer effort anyway so I’ll just jump on it.)
Vendoring specific versions makes sense, it’s entirely understandable you’d not want to target a GNU/Linux distro’s release cadence, especially when they don’t match yours.
I use the packages that you’ve created on Debian with the Stalwart email server.
