As I’ve been walking through pieces of simulation to try and write bits up about how they work, I’ve ended up looking at the testspec again, and realizing that maybe it’s not quite something that I want to use as the described template of how to make a good declarative test spec. TOML is a slightly-more-featureful INI-like language for config files, and testspec is sort of like INI already, so this seems like possibly the most natural transition to something that seems sane to describe?
I’d like to propose adding support reading and running .toml structured test specs in tests/, and then later converting all .txt files to .toml and removing testspec from trunk.
As an example,
testTitle=CloggedCausalConsistencyTest testName=Sideband testDuration=30.0 operationsPerSecond=500 testName=RandomClogging testDuration=30.0 scale=0.1 clogginess=2.0 testName=Attrition machinesToKill=10 machinesToLeave=3 reboot=true testDuration=30.0
[test] title = SimplifiedCloggedCausalConsistencyTest # Other test-wide options like waitForQuiesenceEnd go here [[workload]] testName=Sideband testDuration=30.0 operationsPerSecond=500 [[workload]] testName=RandomClogging testDuration=30.0 scale=0.1 clogginess=2.0 [[workload]] testName=Attrition machinesToKill=10 machinesToLeave=3 reboot=true testDuration=30.0
This raises the question of how would restarting tests work? Restarting tests would try to run
fdbserver-5.2.0 with a TOML testspec, which obviously wouldn’t work. If we would do a complete transition, then there would need to be a small script which consumes a TOML and translates it back into an equivalent testspec, which the test harness would use when running .toml tests on <= 7.0.0. As the change is mostly structural, TOML->testspec conversion should be pretty simple (and almost achievable with sed).
Of the C++ TOML libraries that exist, toml11 (MIT Licensed) is looking like the most supported and easy to use, so I’d like to look into incorporating it into FDB.