Does FoundationDB provide a way to replicate data using a set number of replicas (ie. double, triple) whilst making sure no two replicas are stored in the same DC (assuming the number of DCs is greater than or equal to the number of replicas)?
From what I have gathered from this topic, I think that answer is “no” due to that fact that three_datacenter only support 3 DCs, and multi-region only supports 2 DCs per region (and 2 regions), but I’m looking for a definitive answer. If not, is there any way to implement such a thing? Again, this has likely been covered in the documentation and previous topics but I would just like to clarify. Thanks!
three_datacenter mode does two copies per DC, under the assumption that it’s better to keep a second copy than copy shards over the WAN whenever a storage server fails.
There’s again technically nothing really stopping this from existing, though. One could add a jonahwest_five_dc mode that has a storage policy of 5 dcid x 1 zoneid, which would achieve what you wish. However, you’d still have the problem that whenever one DC fails, data distribution will think every team is failed and not make any useful decisions anymore. Perhaps this is a request for letting one configure custom storage and tLog policies in addition to the named ones…
Generalized multi-region, the thing that doesn’t exist yet, is a much better solution to this than just increasing the dcid count in storage policies. One could do this with FDB, but it’s not a set of use-cases I’d currently go recommending FDB for.
After looking through the docs, could this be achieved by setting the same locality_machineid parameter in each DC?
EDIT: I’ve found locality_zoneid. Is this what I should be using?
--locality_zoneid is what most of the policy logic goes off of, so it is the correct one to set.
Setting zoneid to be a datacenter would let you use triple to spread data across three or more datacenter and synchronously commit to all of them. Your data would only be in three random datacenters, and if you have less than three datacenters, FDB will be unavailable.
I don’t think you’d gain anything by doing this over using one of the modes more meant for multi-dc replication.