# Fdb cluster is unavailable after delete a disk

**URL:** <https://forums.foundationdb.org/t/fdb-cluster-is-unavailable-after-delete-a-disk/2232>\
**Category:** Using FoundationDB\
**Created:** [July 7, 2020, 9:35am UTC](https://forums.foundationdb.org/t/fdb-cluster-is-unavailable-after-delete-a-disk/2232 "2020-07-07T09:35:29Z")\
**Posts on this page:** 1\
**Showing post:** 3

<div class="post-metadata">

**Author:** ![ssb](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/ssb/32/656_2.png) [@ssb](https://forums.foundationdb.org/u/ssb)\
**Post date:** [July 8, 2020, 9:10am UTC](https://forums.foundationdb.org/t/fdb-cluster-is-unavailable-after-delete-a-disk/2232/3 "2020-07-08T09:10:30Z")

</div>

Hello!

When the cluster is unavailable, master role print the follow logs in circle:

> Event Severity=“10” Time=“1594191671.815474” Type=“EndpointNotFound” Machine=“172.16.9.186:4516” ID=“0000000000000000” Address=“172.16.9.186:4506” Token=“6c1737ef6dd69fd9” SuppressedEventCount=“0” logGroup=“default”/\>  
> Event Severity=“10” Time=“1594191671.815474” Type=“CCWDB” Machine=“172.16.9.186:4516” ID=“a7185cdb49a4f1bd” logGroup=“default”/\>  
> Event Severity=“10” Time=“1594191671.815474” Type=“CCWDB” Machine=“172.16.9.186:4516” ID=“a7185cdb49a4f1bd” Recruiting=“Master” logGroup=“default”/\>  
> Event Severity=“10” Time=“1594191671.815885” Type=“CCWDB” Machine=“172.16.9.186:4516” ID=“a7185cdb49a4f1bd” Recruited=“9a9c53aa431aafe1” logGroup=“default”/\>  
> Event Severity=“10” Time=“1594191671.815885” Type=“RecruitedMasterWorker” Machine=“172.16.9.186:4516” ID=“a7185cdb49a4f1bd” Address=“172.16.9.186:4506” logGroup=“default” TrackLatestType=“Original”/\>  
> Event Severity=“10” Time=“1594191671.815885” Type=“CCWDB” Machine=“172.16.9.186:4516” ID=“a7185cdb49a4f1bd” Lifetime=“a7185cdb49a4f1bd#3” ChangeID=“719075b9e13f29b6” logGroup=“default”/\>  
> Event Severity=“10” Time=“1594191671.815885” Type=“GotServerDBInfoChange” Machine=“172.16.9.186:4516” ID=“0000000000000000” ChangeID=“719075b9e13f29b6” MasterID=“9a9c53aa431aafe1” logGroup=“default”/\>  
> Event Severity=“10” Time=“1594191672.012351” Type=“MasterRegistrationReceived” Machine=“172.16.9.186:4516” ID=“a7185cdb49a4f1bd” dbName=“DB” MasterId=“9a9c53aa431aafe1” Master=“zoneid=948936f4795067051c61fe19a5105367 processid=54b460781b6f513d6e3b7665c9135b06 machineid=948936f4795067051c61fe19a5105367” Tlogs=“eeb740bfd8692df182e8d71949785ddc,71d797746a39bbc3f5a88fa1fd866208,19b2db9e2fc2ceb9e00369513ea68011” Resolvers=“0” RecoveryState=“3” RegistrationCount=“1” Proxies=“0” RecoveryCount=“6516” logGroup=“default”/\>  
> Event Severity=“10” Time=“1594191672.012351” Type=“GotServerDBInfoChange” Machine=“172.16.9.186:4516” ID=“0000000000000000” ChangeID=“83637f7243df9578” MasterID=“9a9c53aa431aafe1” logGroup=“default”/\>  
> Event Severity=“10” Time=“1594191672.018168” Type=“GetTLogTeamDone” Machine=“172.16.9.186:4516” ID=“4952dff3ccb92c5c” Completed=“1” Policy=“zoneid^2 x 1” Results=“3” Processes=“3” Workers=“21” Replication=“2” Desired=“3” RatingTests=“200” PolicyGenerations=“100” InterfaceId=“a7185cdb49a4f1bd” logGroup=“default”/\>  
> Event Severity=“10” Time=“1594191671.815885” Type=“GotServerDBInfoChange” Machine=“172.16.9.186:4516” ID=“0000000000000000” ChangeID=“719075b9e13f29b6” MasterID=“9a9c53aa431aafe1” logGroup=“default”/\>  
> Event Severity=“10” Time=“1594191672.012351” Type=“MasterRegistrationReceived” Machine=“172.16.9.186:4516” ID=“a7185cdb49a4f1bd” dbName=“DB” MasterId=“9a9c53aa431aafe1” Master=“zoneid=948936f4795067051c61fe19a5105367 processid=54b460781b6f513d6e3b7665c9135b06 machineid=948936f4795067051c61fe19a5105367” Tlogs=“eeb740bfd8692df182e8d71949785ddc,71d797746a39bbc3f5a88fa1fd866208,19b2db9e2fc2ceb9e00369513ea68011” Resolvers=“0” RecoveryState=“3” RegistrationCount=“1” Proxies=“0” RecoveryCount=“6516” logGroup=“default”/\>  
> Event Severity=“10” Time=“1594191672.012351” Type=“GotServerDBInfoChange” Machine=“172.16.9.186:4516” ID=“0000000000000000” ChangeID=“83637f7243df9578” MasterID=“9a9c53aa431aafe1” logGroup=“default”/\>  
> Event Severity=“10” Time=“1594191672.018168” Type=“GetTLogTeamDone” Machine=“172.16.9.186:4516” ID=“4952dff3ccb92c5c” Completed=“1” Policy=“zoneid^2 x 1” Results=“3” Processes=“3” Workers=“21” Replication=“2” Desired=“3” RatingTests=“200” PolicyGenerations=“100” InterfaceId=“a7185cdb49a4f1bd” logGroup=“default”/\>  
> Event Severity=“10” Time=“1594191672.018168” Type=“GetTLogTeamWorker” Machine=“172.16.9.186:4516” ID=“4952dff3ccb92c5c” Class=“transaction” Address=“178.104.163.99:4505” Zone=“ae5132ed0959f5fbd16edad8584e03ae” DataHall=“[not set]” isExcludedServer=“0” isAvailable=“1” logGroup=“default”/\>  
> Event Severity=“10” Time=“1594191672.018168” Type=“GetTLogTeamWorker” Machine=“172.16.9.186:4516” ID=“4952dff3ccb92c5c” Class=“transaction” Address=“172.16.9.186:4515” Zone=“948936f4795067051c61fe19a5105367” DataHall=“[not set]” isExcludedServer=“0” isAvailable=“1” logGroup=“default”/\>  
> Event Severity=“10” Time=“1594191672.018168” Type=“GetTLogTeamWorker” Machine=“172.16.9.186:4516” ID=“4952dff3ccb92c5c” Class=“transaction” Address=“172.16.9.186:4505” Zone=“948936f4795067051c61fe19a5105367” DataHall=“[not set]” isExcludedServer=“0” isAvailable=“1” logGroup=“default”/\>  
> Event Severity=“10” Time=“1594191672.018168” Type=“findWorkersForConfig” Machine=“172.16.9.186:4516” ID=“0000000000000000” replication=“2” desiredLogs=“3” actualLogs=“3” desiredProxies=“3” actualProxies=“3” desiredResolvers=“1” actualResolvers=“1” logGroup=“default”/\>  
> Event Severity=“10” Time=“1594191672.819949” Type=“CCWDB” Machine=“172.16.9.186:4516” ID=“a7185cdb49a4f1bd” Watching=“9a9c53aa431aafe1” logGroup=“default”/\>  
> Event Severity=“10” Time=“1594191672.820138” Type=“EndpointNotFound” Machine=“172.16.9.186:4516” ID=“0000000000000000” Address=“172.16.9.186:4506” Token=“0d972419b4b545fd” SuppressedEventCount=“0” logGroup=“default”/\>

From the log, we know 172.16.9.186:4506 endpoint is not found(it run on the deleted disk), but why master role select an failed endpoint every time?

UPD1: If there are 3 machines, and test the same case, the cluster is available. According to [the thread](https://forums.foundationdb.org/t/full-disk-on-one-machine-results-in-99-performance-degradation/850) in your reply, the cluster should be unavailable too.

UPD2: I test the same case on foundationdb of version 6.2.19, the cluster is available.

---

_[View the full topic](https://forums.foundationdb.org/t/fdb-cluster-is-unavailable-after-delete-a-disk/2232)._