5 ms·
One of the leveldb authors here. TokyoCabinet is something we seriously considered using instead of writing leveldb. TokyoCabinet has great performance usuall
by sghemawat 15y ago
One of the leveldb authors here. TokyoCabinet is something we seriously considered using instead of writing leveldb. TokyoCabinet has great performance usually. I haven't done a careful head-to-head comparison, but it wouldn't surprise me if it was somewhat faster than leveldb for many workloads. Plus TokyoCabinet is more mature, has matching server code etc. and may therefore be a better fit for many projects.
However because of a fundamental difference in data structures (TokyoCabinet uses btrees for ordered storage; leveldb uses log structured merge trees), random write performance (which is important for our needs) is significantly better in leveldb. This part we did measure. IIRC, we could fill TokyoCabinet with a million 100-byte writes in less than two seconds if writing sequentially, but the time ballooned to ~2000 seconds if we wrote randomly. The corresponding slowdown for leveldb is from ~1.5 seconds (sequential) to ~2.5 seconds (random).
- lisper 15y agoDid you consider fractal trees? http://tokutek.com/presentations/bender-Scalperf-9-09.pdf http://tokutek.com/presentations/bender-Scalperf-9-09.pdf
- psaccounts 15y agoIs there a more useful technical paper on Fractal Trees? Better yet is there a open source implementation of the same?
- jbapple 15y agoCache-oblivious streaming B-trees: http://supertech.csail.mit.edu/papers/sbtree.pdf http://supertech.csail.mit.edu/papers/sbtree.pdf I don't know for certain any open-source implementation, but I have heard COLAs are used in HBase.
- yingfeng 15y agofractal trees should have a faster read performance with leveldb with a comparable updating performance. There does not exist any open source solution for such structure, while we could hope that www.acunu.com might provide an open source solution for similary Stratified B-tree http://www.acunu.com/2011/04/log-file-systems-and-ssds-made-for-each-other/ http://www.acunu.com/2011/04/log-file-systems-and-ssds-made-...
- br1 15y agoBeing in-kernel, acunu is very far from the lightness of LevelDB.
- spullara 15y agoHave you looked at implementing CAS? It is the one thing missing for me to adopt it as a underlying store for my AvroBase API: http://www.javarants.com/2010/06/30/havrobase-a-searchable-evolvable-entity-store-on-top-of-hbase-and-solr/ http://www.javarants.com/2010/06/30/havrobase-a-searchable-e...
- davidhollander 15y ago> difference in data structures Always good to have more tools in one's arsenal in that case. I'll drop one of you a message if I write a Lua binding for it (using LuaJIT+embedded k/v for data services atm).
- cmartella 15y agoCould you please report about caching setup for random read test? was there any cache (a part of OS's) for either position/index lookup or payload? Or did you hit the disk without any caching of past operations? thanks