# RocksDB backend **URL:** \ **Category:** Development\ **Created:** [November 5, 2018, 1:22pm UTC](https://forums.foundationdb.org/t/rocksdb-backend/845 "2018-11-05T13:22:47Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![doublemax](https://avatars.discourse-cdn.com/v4/letter/d/8e7dd6/32.png) [@doublemax](https://forums.foundationdb.org/u/doublemax)\ **Post date:** [November 5, 2018, 1:22pm UTC](https://forums.foundationdb.org/t/rocksdb-backend/845/1 "2018-11-05T13:22:47Z")
Are there any plans to support a LSM tree like “RocksDB” as compile-time option? ---
**Author:** ![yangruifeng](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/yangruifeng/32/710_2.png) [@yangruifeng](https://forums.foundationdb.org/u/yangruifeng)\ **Post date:** [November 6, 2018, 7:23am UTC](https://forums.foundationdb.org/t/rocksdb-backend/845/2 "2018-11-06T07:23:03Z")
hi，here is a demo，which use rocksdb as backend。 [rocksdb backend for foundationdb](https://github.com/yangruifeng/foundationdb/tree/yrf) ---
**Author:** ![ryanworl](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/ryanworl/32/440_2.png) [@ryanworl](https://forums.foundationdb.org/u/ryanworl)\ **Post date:** [November 6, 2018, 12:20pm UTC](https://forums.foundationdb.org/t/rocksdb-backend/845/3 "2018-11-06T12:20:36Z")
> [@Discussion thread for new storage engine ideas](https://forums.foundationdb.org/t/discussion-thread-for-new-storage-engine-ideas/101/7): > > When integrating with off the shelf storage engines that do I/O synchronously (including via memory mapped files) I think some effort may be needed to get decent performance. FoundationDB uses a single-threaded asynchronous execution model and blocking that thread to read from disk will be disastrous for throughput and latency. Our adaptation of sqlite uses coroutines to let it use our asynchronous I/O abstraction. That approach won’t work for memory mapped I/O. You could use a real thread p… This post explains why RocksDB, and other existing storage engines, are not included as options. They do not work with the simulation testing framework as they require thread pools because they do synchronous IO. The simulation testing framework requires that every part of the database be able to run in a single thread. That doesn’t mean you can’t or shouldn’t write an adapter for RocksDB if you personally trust it, but you would need to do the testing yourself to verify things worked as expected. ---
**Author:** ![doublemax](https://avatars.discourse-cdn.com/v4/letter/d/8e7dd6/32.png) [@doublemax](https://forums.foundationdb.org/u/doublemax)\ **Post date:** [November 6, 2018, 1:05pm UTC](https://forums.foundationdb.org/t/rocksdb-backend/845/4 "2018-11-06T13:05:22Z")
@ [ryanworl](https://forums.foundationdb.org/u/ryanworl) Thanks for pointing out. ---
**Author:** ![davelester](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/davelester/32/1927_2.png) [@davelester](https://forums.foundationdb.org/u/davelester)\ **Post date:** [November 6, 2018, 2:59pm UTC](https://forums.foundationdb.org/t/rocksdb-backend/845/5 "2018-11-06T14:59:19Z")
Any plans to contribute this back to the OSS project, or thoughts on whether you’re looking to maintain this outside of the demo? ---
**Author:** ![doublemax](https://avatars.discourse-cdn.com/v4/letter/d/8e7dd6/32.png) [@doublemax](https://forums.foundationdb.org/u/doublemax)\ **Post date:** [November 6, 2018, 7:04pm UTC](https://forums.foundationdb.org/t/rocksdb-backend/845/6 "2018-11-06T19:04:11Z")
@[yangruifeng](https://forums.foundationdb.org/t/rocksdb-backend/845/2) How did you solve the async issue? A dedicated thread? The async-patch was never applied to RocksDB. ---
**Author:** ![alexmiller](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/alexmiller/32/326_2.png) [@alexmiller](https://forums.foundationdb.org/u/alexmiller)\ **Post date:** [November 6, 2018, 7:47pm UTC](https://forums.foundationdb.org/t/rocksdb-backend/845/7 "2018-11-06T19:47:06Z")
@yangruifeng, is this a thing that you’d be interested in trying to see if there’s a way to adapt RocksDB to run in a more evented style and integrated into the main FDB run loop so that it can get merged into mainline FDB? For the sqlite storage engine, there’s a layer that adapts stackful coroutines into flow, [CoroFlow.actor.cpp](https://github.com/apple/foundationdb/blob/6bb1f4093d9b363a350831341230c6cb46e3c906/fdbserver/CoroFlow.actor.cpp), that you might be able to use to similarly flow-ify RocksDB operations. But like, well done though. I’m really happy to see that integrating in a new storage engine wasn’t too much work, and was feasible to do without help and guidance. > [@doublemax](#): > > A dedicated thread? The code looks like all operations are tossed onto a rocksdb::ThreadPool. ---
**Author:** ![doublemax](https://avatars.discourse-cdn.com/v4/letter/d/8e7dd6/32.png) [@doublemax](https://forums.foundationdb.org/u/doublemax)\ **Post date:** [November 6, 2018, 7:48pm UTC](https://forums.foundationdb.org/t/rocksdb-backend/845/8 "2018-11-06T19:48:31Z")
“wiredtiger” (also LSM) would supports async ops - but I don’t know if range-deletes are possible. ---
**Author:** ![yangruifeng](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/yangruifeng/32/710_2.png) [@yangruifeng](https://forums.foundationdb.org/u/yangruifeng)\ **Post date:** [November 7, 2018, 2:47am UTC](https://forums.foundationdb.org/t/rocksdb-backend/845/9 "2018-11-07T02:47:26Z")
@doublemax @alexmiller yes, a decicated threadpool(rocksdb::ThreadPool). @alexmiller do you mean like this? this is my old design, but I worry that rocksdb **write stalls** affects main thread. ![image](https://global.discourse-cdn.com/foundationdb/original/1X/9e3318e88543f9ef550fff0395138b8628e9ea07.png) @davelester the coding is rough, any suggestion is welcome, then i can contibute this back to the OSS project. ---
**Author:** ![ricky.saltzer](https://avatars.discourse-cdn.com/v4/letter/r/f17d59/32.png) [@ricky.saltzer](https://forums.foundationdb.org/u/ricky.saltzer)\ **Post date:** [November 8, 2018, 10:44pm UTC](https://forums.foundationdb.org/t/rocksdb-backend/845/10 "2018-11-08T22:44:31Z")
Would there be any benefit to also having an LMDB based backend as well? ---
**Author:** ![alexmiller](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/alexmiller/32/326_2.png) [@alexmiller](https://forums.foundationdb.org/u/alexmiller)\ **Post date:** [November 9, 2018, 7:45am UTC](https://forums.foundationdb.org/t/rocksdb-backend/845/11 "2018-11-09T07:45:19Z")
There was a RocksDB meetup today, and Nutanix presented some very similar work that they did in dispatching RocksDB reads and writes from a pool of user-level threads / fibers. I’m hoping slides will show up soon that I can point you towards. Your diagram is roughly a layering of what I’d expect this to look like. You’re going to be following a general pattern of getting RocksDB operations to run in a coroutine, and for any call that they would make to block, call into Flow code that returns a future as to when the coroutine can resume, and then suspend the coroutine and switch to something new. CoroFlow, and its existing usage by sqlite, should be of help in figuring out how to do that. I think the rough concrete outline of work would be to define a new `RocksDB::Env` that implements: 1. most functions with their `platform::*` equivalent 2. a new implementation of `SequentialFile` and `RandomAccessFile` that wrap `IAsyncFile` 3. `schedule()` as a function that spawns a new coroutine And then see what fails. I’d probably start by defining a Flow unit test that initializes a FlowRocksDBEnv and calls `get()` on a random key, and start implementing the minimal things you need to get that working. This strongly relies upon the assumption that all blocking operations that RocksDB does somehow goes through `RocksDB::Env`. Anything from memory-mapped files, to sleep() calls, to mutex usage, would invalidate this assumption, and probably make integrating RocksDB into deterministic simulation infeasible. * * * The above defines the work that would need to be done in order to integrate RocksDB into FDB’s deterministic simulation tests. For running with RocksDB on real world clusters, I’m concerned that trying to pack the CPU load of compactions onto the same thread as what serves user read requests would be impractical. I suspect that although packing everything into one thread will be needed for testing, we’ll need to run with compactions running on threads in the background, but we can figure that out once it’s shown that we can solve the testing side of this problem. ---
**Author:** ![doublemax](https://avatars.discourse-cdn.com/v4/letter/d/8e7dd6/32.png) [@doublemax](https://forums.foundationdb.org/u/doublemax)\ **Post date:** [November 11, 2018, 6:15pm UTC](https://forums.foundationdb.org/t/rocksdb-backend/845/12 "2018-11-11T18:15:06Z")
“SQLite4” is also a LSM (with fast range-delete). Maybe FDB should simply change to SQLite4? [https://sqlite.org/src4/doc/trunk/www/lsmusr.wiki](https://sqlite.org/src4/doc/trunk/www/lsmusr.wiki) ---
**Author:** ![KrzysFR](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/krzysfr/32/43_2.png) [@KrzysFR](https://forums.foundationdb.org/u/KrzysFR)\ **Post date:** [November 11, 2018, 8:52pm UTC](https://forums.foundationdb.org/t/rocksdb-backend/845/13 "2018-11-11T20:52:36Z")
It seems that work on sqlite4’s LSM engine has stopped for some time now: Last commit: [https://sqlite.org/src4/info/c0b7f14c0976ed5e](https://sqlite.org/src4/info/c0b7f14c0976ed5e) ---
**Author:** ![yangruifeng](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/yangruifeng/32/710_2.png) [@yangruifeng](https://forums.foundationdb.org/u/yangruifeng)\ **Post date:** [November 12, 2018, 2:09am UTC](https://forums.foundationdb.org/t/rocksdb-backend/845/14 "2018-11-12T02:09:41Z")
LSM engine has been folded into SQLite3 ---
**Author:** ![doublemax](https://avatars.discourse-cdn.com/v4/letter/d/8e7dd6/32.png) [@doublemax](https://forums.foundationdb.org/u/doublemax)\ **Post date:** [November 12, 2018, 8:43am UTC](https://forums.foundationdb.org/t/rocksdb-backend/845/15 "2018-11-12T08:43:39Z")
You are right. The LSM was merged into SQLite3 (interesting). A list of possible storage engines: github. com/pmwkaa/engine.so The LSM trees are: SQLite3, LevelDB/RocksDB and WiredTiger ---
**Author:** ![doublemax](https://avatars.discourse-cdn.com/v4/letter/d/8e7dd6/32.png) [@doublemax](https://forums.foundationdb.org/u/doublemax)\ **Post date:** [November 15, 2018, 7:24pm UTC](https://forums.foundationdb.org/t/rocksdb-backend/845/16 "2018-11-15T19:24:35Z")
Maybe it is easier to enable the LSM tree of SQLite? [https://www.sqlite.org/cgi/src/dir?ci=9b37bbf5f338dea9&name=ext/lsm1](https://www.sqlite.org/cgi/src/dir?ci=9b37bbf5f338dea9&name=ext/lsm1)