Recently I’ve known the testing technique used in FoundationDB, which is very amazing to me. What I’ve been pursuing is a kind of testing framework that can effectively detect/reproduce bugs in distributed system. Obviously FDB has done lots of work about this.
I have some questions to ask:
- In your team’s testing environment, how many machines you’re willing to offer to support such a big testing, especially concurrent testing triggered by multiple developers (considering isolation of testing among them) ?
- Once a developer pushes a commit, will all of FDB test cases (I mean all) be ran ? I think it will take lots of time when the commit is considered OK by all test cases. Is this acceptable for everyone ?
- How long does finishing all test cases take ?
To sum up, I’m concerned about time and resources taken by such a rigorous and complex testing during everyone’s daily development.