Additional sidecar containers (for exporting trace logs) with the Kubernetes operator?

I’m trying to figure out how to get trace logs from fdbserver processes running in Kubernetes with the FDB Kubernetes operator to a remote logging aggregator. I found an older thread discussing the same issue, and the punchline there was:

The operator sends the trace logs from the fdbserver processes to a directory that is mapped to an emptyDir volume. This is designed so that if you want to export the trace logs to another system, you could add a custom sidecar container that accesses that volume and exports the logs.

That makes sense to me on the surface, but how can we actually add a custom sidecar container? The most plausible-sounding hook I’ve found so far is podTemplate in ProcessSettings, but that says:

If a container image with a tag is specified the operator will throw an error and stop processing the cluster.

…so it sounds like that’s meant to discourage exactly what I’m trying to do.

Given the constraints, what’s the best way to add a sidecar container to export trace logs? Or is the idea that we replace the existing sidecar image with our own custom image?

Thanks!

You were on the right track with adding an additional container with the ProcessSettings, we actually have an example in our docs: fdb-kubernetes-operator/docs/manual/customization.md at main · FoundationDB/fdb-kubernetes-operator · GitHub. The statement

If a container image with a tag is specified the operator will throw an error and stop processing the cluster.

Was referring to the main container and sidecar container as those versions (tags) are managed by the operator. We should adjust the wording here. Could you open a GitHub issue to improve the wording here (and if you want you can also file a PR, happy to review it).

1 Like

Excellent; thanks kindly! I’ll follow up with an issue shortly and a PR as soon as I’m confident in my understanding.