6 ms·
Datafusion has all of the individual pieces, especially if you just want to data analysis, but is far from an embedded database. Indexes, transactions, a first
by hobofan 21d ago
Datafusion has all of the individual pieces, especially if you just want to data analysis, but is far from an embedded database.
Indexes, transactions, a first-class storage format are all things that come included with DuckDB, that you won't have with Datafusion.
- andriy_koval 21d agoindexes are very weak in duckdb, need to fit memory.
- 1egg0myegg0 21d agoThis fall, DuckDB will support larger than memory indexes! I'm really excited about it. https://duckdb.org/2026/08/17/duckdb-20-highlights#7-storage-format-v20 https://duckdb.org/2026/08/17/duckdb-20-highlights#7-storage...
- andriy_koval 21d agothey don't say "this fall", they say later this year, I think it is some parallel effort, and it is actually going for few years already (rewriting index logic to have them in mmapped buffers and not in memory), finger crossed they will finish it finally.