Hello FoundationDB community,
I’m running a FoundationDB cluster on Kubernetes using the official FoundationDB operator. Currently, I need to expose the cluster to external clients (outside the K8s cluster), ideally using ServiceType=LoadBalancer. But operator only support ClusterIP.
What way for exposing cluster exists?
If required you could add support for ServiceType=LoadBalancer, but depending on your Kubernetes solution and the size of your FDB cluster this might be pretty expensive and will add additional network hops. If you have control over the Kubernetes cluster, it’s probably easier to implement a solution where the Pod IPs are reachable from the outside of the Kubernetes cluster, some CNI solutions support that.
Thanks, I’m trying to do that now. But I’m having some trouble reconcile it.
The cluster isn’t very large, 20-40 pods. And our infrastructure doesn’t want to reconfigure the network 
Bear in mind any FDB client needs to directly access many (all?) nodes of the cluster. So unless I’ve misunderstood you’d need a service per pod. The default clusterip service is headless (doesn’t actually have an IP of its own at all), it’s just there as a means to reference pods by a unique hostname that includes the service name, namespace etc, instead of by IP. Even if you gave it an IP you couldn’t use that IP as the only route to talk to the cluster from a client.
Maybe it’s already possible to get the operator to deploy a clusterip service per pod, and assign each a unique service IP, and I’ve just missed it. But that would seem a bit pointless when the pod IPs should be routable in-cluster already.
The pods would then need be configured to each know and advertise their external IP/hostname as well, so that connecting clients use those when discovering cluster topology from the coordinators, not the internal IPs.
Datomic also had this similar pattern/issue, and their response for ‘Datomic Cloud’ was to write a proxy component that sat alongside the cluster and exposed an HTTP API which could be used client-side via a similar SDK to the ‘core’ client. That proxy connected to all the backends, commit servers etc, and clients only connected to the proxy.
AFAIK no such proxy exists for FDB, so you’d have to write it.