Jepsen testing of FoundationDB

Using FoundationDB
1

I’ve been wondering if there are any plans to get Kyle Kingsbury to perform Jepsen testing of FoundationDB?

I do realize that Kyle anectodally said that he would not test FoundationDB because it was developed using more thorough tests than the Jepsen suite, but as Jepsen is becoming a common ground for distributed databases, it would be great to be able to point to a test report.

I am in fact now writing a note about possible choices for a distributed database and using Jepsen analyses to eliminate many unreliable options. I then have to explain that the lack of test results for FoundationDB is not necessarily disqualifying.

2 Likes
2

Seems like Jennifer Rullmann did some work towards that. Not sure what happened after.

She also did some work towards another test suite targeting the isolation levels. FoundationDB did pretty good with default configuration.

Yeah it will be nice to have an official post from Jepsen.

3

I think it would be great but someone would have to fund it. Hard when the top users of it already know that it is reliable and it isn’t necessary to prove it to them.

4

There was at least one blog post, perhaps two, detailing the test results, which showed FoundationDB passing all Jepsen tests with flying colors. One could hunt around in the Internet Archive for them.

5

I found Call Me Maybe: FoundationDB vs Jepsen and Running the New Jepsen Test.

I hadn’t ever actually gone and looked at the old blog before. There’s a number of good posts on there.

4 Likes
6

These blog posts are great! I somehow completely missed them, and they contain a lot of useful information. Perhaps if there is no way to sponsor Jepsen testing, these posts could be resurrected and made more visible?

7

Reviving an old thread, as the issue of FoundationDB not having a Jepsen analysis comes up for me every once in a while. I tell people to look at Jepsen to eliminate unreliable choices, and then inevitably get the question of why FoundationDB isn’t there.

I did ask @jepsen_io on Twitter:

2020-05-28 at 12.45
2020-05-28 at 12.45904×138 31.3 KB

Which is something I’d agree with :slight_smile:

1 Like
8

I guess the harder problem is what is the motivation to do that, since Apple and other data companies have been using FDB in production for years. :wink:

Just thinking loud: maybe people and companies who are interested in Jepsen test can chip in? :slight_smile:

1 Like
9

Well, Jepsen discovered multiple problems even with databases that have been used for years in multiple production setups, so the value is still there. I know FDB is exceptional in that it is architected with testing first, but there is still value in independent verification. And a good result would really help adoption.

1 Like