A FoundationDB layer for apiserver as an alternative to etcd

You might be interested in this thread that @PierreZ started a while ago about the etcd API: Multiple questions about Indexes, functions and watches to implement etcd-layer

Yeah, that’s right. What FDB calls “watch” is a feature that lets you know if a specific key has changed, but it doesn’t tell you what it was changed to. The etcd “watch” API gives you a list of all changes to the specified key range (until the watch is ended). I think, with the right indexes, you could probably use FDB watches to implement etcd watches, but I’m eliding over a lot of implementation details there.

Hmm, I’m not sure I completely understand the question. It’s certainly possible to create read hot spots in FDB, and if the idea is to implement the “version” in MVCC by including a single key which is incremented every time a change is made, then yes, that key would quickly become a hotspot. I think most proposals for putting something like etcd on top of FDB would be to leverage FDB’s existing “version” in some way. Each FDB cluster already maintains a monotonically increasing 64-bit version that it uses for MVCC (though note that the MVCC history window is not persisted, and so it can only be queried historically for 5 seconds, i.e., the duration the MVCC history window is available in memory), and it also exposes operations to include the version in keys and values (which is equivalent to, say, including the “commit timestamp” in a column in an alternative system). Then I think you could build persistent MVCC into the layer’s data model with those tools. But it would require some data modeling work to get right.

2 Likes