As we know, user can define the numbers of different FDB processes. And all the required roles will be allocated among the defined processes.
Can operator control the role allocation right now? For example, I define a cluster with 3 proxy and 4 stateless processes. Can the “commit_proxy” and “grv_proxy” role only go to the proxy processes instead of any stateless process? In my testing, I found even if I have one proxy process idle, one of the stateless processes got a “grv_proxy” role.
The operator is not doing the actual assignment. The operator is only doing the configuration via. the configure command and the assignment is done by FDB itself. Based on foundationdb/Locality.cpp at 7.1.25 · apple/foundationdb · GitHub it looks like what you are trying to achieve should be supported.
Do you mind to share some more information like FDB version (I assume 7.1, since you’re using commit and grv proxies), the FoundationDBCluster spec for the operator and the Foundationdb cluster status (can be fetched with `fdbcli --exec ‘status json’ inside a Pod).
Is there a reason you want to separate the stateless processes and the proxies?
The issue seems to be that in 7.1, the role of proxy split into two. Thus, what used to be asking for 3 proxy hosts will need to change to grv_proxies and commit_proxies. The thing is, the proxy is also needed for communication and how to identify the proxy pod that is open for communication?
We have added creating LoadBalancer to enable remote access to FDB cluster in cloud. Since external service IP is limited resource, we just create LB for those needed by remote visit. So we didn’t create LB for stateless processes while we did for proxy ones. Now in FDB 7.1, proxy are split into commit_proxy and grv_proxy and seems grv_proxy prefer stateless process instead of proxy process, we need to think about which processes need LB.
We have added creating LoadBalancer to enable remote access to FDB cluster in cloud. Since external service IP is limited resource, we just create LB for those needed by remote visit. So we didn’t create LB for stateless processes while we did for proxy ones. Now in FDB 7.1, proxy are split into commit_proxy and grv_proxy and seems grv_proxy prefer stateless process instead of proxy process, we need to think about which processes need LB.
Based on the code I shared above it should work. Like I said if you could share you configuration that would be useful.