RocksDB backend

[ryanworl]

Discussion thread for new storage engine ideas

When integrating with off the shelf storage engines that do I/O synchronously (including via memory mapped files) I think some effort may be needed to get decent performance. FoundationDB uses a single-threaded asynchronous execution model and blocking that thread to read from disk will be disastrous for throughput and latency. Our adaptation of sqlite uses coroutines to let it use our asynchronous I/O abstraction. That approach won’t work for memory mapped I/O. You could use a real thread p…

This post explains why RocksDB, and other existing storage engines, are not included as options. They do not work with the simulation testing framework as they require thread pools because they do synchronous IO. The simulation testing framework requires that every part of the database be able to run in a single thread.

That doesn’t mean you can’t or shouldn’t write an adapter for RocksDB if you personally trust it, but you would need to do the testing yourself to verify things worked as expected.