When execute fix-coordinator-ips got "Could not find process for coordinator IP.." error

Using FoundationDB Kubernetes Operator
1

If there is coordinator ip changes, like I delete more than half pods, it will trigger this recover step, and when trying to execute the fix-coordinator-ips command, got error

sh-4.4$ kubectl fdb fix-coordinator-ips -c mdm-foundationdb-ibm -n testoperator1
2022/07/27 06:46:23 Could not find process for coordinator IP 10.254.14.174:4500:tls
2022/07/27 06:46:23 Could not find process for coordinator IP 10.254.23.103:4500:tls
2022/07/27 06:46:23 Could not find process for coordinator IP 10.254.17.99:4500:tls
2022/07/27 06:46:23 New connection string: mdm_foundationdb_ibm:hXU4v6hVgaKKYRgrrrH7JIdfX2Trbp4a@10.254.14.174:4500:tls,10.254.23.103:4500:tls,10.254.17.99:4500:tls

could you share some insights what might be wrong?

2

I assume by delete you mean you run kubectl delete po .. or the equivalent in you favourite programming language? Have you ensured that the new Pods are created and running? This is the place where the error is thrown: fdb-kubernetes-operator/fix_coordinator_ips.go at main · FoundationDB/fdb-kubernetes-operator · GitHub. If you are able to reproduce this issue, could you open a GitHub issue with a minimal reproducible setup? It would also be helpful if you could share the cluster status from the cluster definition in Kubernetes.

3

@johscheuer I opened Execute fix-coordinator-ips command got “Could not find process for coordinator IP..” error · Issue #1314 · FoundationDB/fdb-kubernetes-operator · GitHub let me know if you need more info. thank you.