Hello!
When the cluster is unavailable, master role print the follow logs in circle:
Event Severity=“10” Time=“1594191671.815474” Type=“EndpointNotFound” Machine=“172.16.9.186:4516” ID=“0000000000000000” Address=“172.16.9.186:4506” Token=“6c1737ef6dd69fd9” SuppressedEventCount=“0” logGroup=“default”/>
Event Severity=“10” Time=“1594191671.815474” Type=“CCWDB” Machine=“172.16.9.186:4516” ID=“a7185cdb49a4f1bd” logGroup=“default”/>
Event Severity=“10” Time=“1594191671.815474” Type=“CCWDB” Machine=“172.16.9.186:4516” ID=“a7185cdb49a4f1bd” Recruiting=“Master” logGroup=“default”/>
Event Severity=“10” Time=“1594191671.815885” Type=“CCWDB” Machine=“172.16.9.186:4516” ID=“a7185cdb49a4f1bd” Recruited=“9a9c53aa431aafe1” logGroup=“default”/>
Event Severity=“10” Time=“1594191671.815885” Type=“RecruitedMasterWorker” Machine=“172.16.9.186:4516” ID=“a7185cdb49a4f1bd” Address=“172.16.9.186:4506” logGroup=“default” TrackLatestType=“Original”/>
Event Severity=“10” Time=“1594191671.815885” Type=“CCWDB” Machine=“172.16.9.186:4516” ID=“a7185cdb49a4f1bd” Lifetime=“a7185cdb49a4f1bd#3” ChangeID=“719075b9e13f29b6” logGroup=“default”/>
Event Severity=“10” Time=“1594191671.815885” Type=“GotServerDBInfoChange” Machine=“172.16.9.186:4516” ID=“0000000000000000” ChangeID=“719075b9e13f29b6” MasterID=“9a9c53aa431aafe1” logGroup=“default”/>
Event Severity=“10” Time=“1594191672.012351” Type=“MasterRegistrationReceived” Machine=“172.16.9.186:4516” ID=“a7185cdb49a4f1bd” dbName=“DB” MasterId=“9a9c53aa431aafe1” Master=“zoneid=948936f4795067051c61fe19a5105367 processid=54b460781b6f513d6e3b7665c9135b06 machineid=948936f4795067051c61fe19a5105367” Tlogs=“eeb740bfd8692df182e8d71949785ddc,71d797746a39bbc3f5a88fa1fd866208,19b2db9e2fc2ceb9e00369513ea68011” Resolvers=“0” RecoveryState=“3” RegistrationCount=“1” Proxies=“0” RecoveryCount=“6516” logGroup=“default”/>
Event Severity=“10” Time=“1594191672.012351” Type=“GotServerDBInfoChange” Machine=“172.16.9.186:4516” ID=“0000000000000000” ChangeID=“83637f7243df9578” MasterID=“9a9c53aa431aafe1” logGroup=“default”/>
Event Severity=“10” Time=“1594191672.018168” Type=“GetTLogTeamDone” Machine=“172.16.9.186:4516” ID=“4952dff3ccb92c5c” Completed=“1” Policy=“zoneid^2 x 1” Results=“3” Processes=“3” Workers=“21” Replication=“2” Desired=“3” RatingTests=“200” PolicyGenerations=“100” InterfaceId=“a7185cdb49a4f1bd” logGroup=“default”/>
Event Severity=“10” Time=“1594191671.815885” Type=“GotServerDBInfoChange” Machine=“172.16.9.186:4516” ID=“0000000000000000” ChangeID=“719075b9e13f29b6” MasterID=“9a9c53aa431aafe1” logGroup=“default”/>
Event Severity=“10” Time=“1594191672.012351” Type=“MasterRegistrationReceived” Machine=“172.16.9.186:4516” ID=“a7185cdb49a4f1bd” dbName=“DB” MasterId=“9a9c53aa431aafe1” Master=“zoneid=948936f4795067051c61fe19a5105367 processid=54b460781b6f513d6e3b7665c9135b06 machineid=948936f4795067051c61fe19a5105367” Tlogs=“eeb740bfd8692df182e8d71949785ddc,71d797746a39bbc3f5a88fa1fd866208,19b2db9e2fc2ceb9e00369513ea68011” Resolvers=“0” RecoveryState=“3” RegistrationCount=“1” Proxies=“0” RecoveryCount=“6516” logGroup=“default”/>
Event Severity=“10” Time=“1594191672.012351” Type=“GotServerDBInfoChange” Machine=“172.16.9.186:4516” ID=“0000000000000000” ChangeID=“83637f7243df9578” MasterID=“9a9c53aa431aafe1” logGroup=“default”/>
Event Severity=“10” Time=“1594191672.018168” Type=“GetTLogTeamDone” Machine=“172.16.9.186:4516” ID=“4952dff3ccb92c5c” Completed=“1” Policy=“zoneid^2 x 1” Results=“3” Processes=“3” Workers=“21” Replication=“2” Desired=“3” RatingTests=“200” PolicyGenerations=“100” InterfaceId=“a7185cdb49a4f1bd” logGroup=“default”/>
Event Severity=“10” Time=“1594191672.018168” Type=“GetTLogTeamWorker” Machine=“172.16.9.186:4516” ID=“4952dff3ccb92c5c” Class=“transaction” Address=“178.104.163.99:4505” Zone=“ae5132ed0959f5fbd16edad8584e03ae” DataHall=“[not set]” isExcludedServer=“0” isAvailable=“1” logGroup=“default”/>
Event Severity=“10” Time=“1594191672.018168” Type=“GetTLogTeamWorker” Machine=“172.16.9.186:4516” ID=“4952dff3ccb92c5c” Class=“transaction” Address=“172.16.9.186:4515” Zone=“948936f4795067051c61fe19a5105367” DataHall=“[not set]” isExcludedServer=“0” isAvailable=“1” logGroup=“default”/>
Event Severity=“10” Time=“1594191672.018168” Type=“GetTLogTeamWorker” Machine=“172.16.9.186:4516” ID=“4952dff3ccb92c5c” Class=“transaction” Address=“172.16.9.186:4505” Zone=“948936f4795067051c61fe19a5105367” DataHall=“[not set]” isExcludedServer=“0” isAvailable=“1” logGroup=“default”/>
Event Severity=“10” Time=“1594191672.018168” Type=“findWorkersForConfig” Machine=“172.16.9.186:4516” ID=“0000000000000000” replication=“2” desiredLogs=“3” actualLogs=“3” desiredProxies=“3” actualProxies=“3” desiredResolvers=“1” actualResolvers=“1” logGroup=“default”/>
Event Severity=“10” Time=“1594191672.819949” Type=“CCWDB” Machine=“172.16.9.186:4516” ID=“a7185cdb49a4f1bd” Watching=“9a9c53aa431aafe1” logGroup=“default”/>
Event Severity=“10” Time=“1594191672.820138” Type=“EndpointNotFound” Machine=“172.16.9.186:4516” ID=“0000000000000000” Address=“172.16.9.186:4506” Token=“0d972419b4b545fd” SuppressedEventCount=“0” logGroup=“default”/>
From the log, we know 172.16.9.186:4506 endpoint is not found(it run on the deleted disk), but why master role select an failed endpoint every time?
UPD1: If there are 3 machines, and test the same case, the cluster is available. According to the thread in your reply, the cluster should be unavailable too.
UPD2: I test the same case on foundationdb of version 6.2.19, the cluster is available.