Only seven processes are online?

Using FoundationDB
1

I’m testing foundationdb on five machines and using triple ssd mode. And for every machine, I have two processes.

But strangely no matter how I configure the cluster, there are only 7 processes online. Let’s assume the IPs are 01 , 02 , 03 , 04 , 05 . So all processes should be:

01:4500
01:4501
02:4500
02:4501
03:4500
03:4501
04:4500
04:4501
05:4500
05:4501

But there are always three random processes that don’t appear in the status details . Although there is at least one process available on every machine.

I expect that all 10 processes should be available. And I’ve checked the processes on every machine by ps -ef . All of them are working correctly.

So what could be the problem?

2

Do any of your processes have severity 40 events in the trace logs?

Another possibility I’ve seen in the past happened when the processes were started with the same contents in their processId files, which are stored in the data directory for each process. Is there any chance that this file was copied between the processes in your deployment? If so, I think only one process with each ID can join. You can resolve this by deleting all of these processId files and letting new ones be created. See How are 'contributing_workers' computed?.

3

Thanks for your help. It looks like to be the issue. After recreating every processId, now it works correctly.

However, I just followed the instruction of build the cluster document. It says it should be better to configure one foundationdb and clone the VM to create the cluster. But the fact is that cloning VM will also create duplicated processId, correct?

4

Correct. Would you mind linking me where the documentation says to do that…?

5

I’m sorry after re-reading the lines multiple times I found I totally reversed the meaning. It actually says don’t do this. :sweat: