ankoh

5 ms·

1.
by ankoh 4y ago
This has to be ruled out first: https://github.com/whatwg/fs/issues/7#issuecomment-116176851... ...but then the OPFS will be a quite decent fit. We (DuckDB-Wasm) are also looking closely at OPFS. IMHO the req
2.
by ankoh 4y ago
We're still on a journey to explore what APIs work best with JavaScript but the differences between WASM and Node are on purpose. DuckDB-Wasm has isolated wasm heap memory and runs in a separate web worker. That's why we serialize
3.
by ankoh 5y ago
The author outlines many problems that you'll run into when implementing a persistent storage backend using the current browser APIs. We faced many of them ourselves but paused any further work on an IndexedDB-backend due to the lack o
4.
by ankoh 5y ago
I agree! DuckDB-Wasm can already open DuckDB database files in the browser the very same way.
5.
by ankoh 5y ago
It depends. Querying CSV files is particularly painful over the network since we still have to read everything for a full scan. With Parquet, you would at least only have to read the columns of group by keys and aggregate arguments. Try it
6.
by ankoh 5y ago
DuckDB-Wasm uses a traditional buffer manager and evicts pages using a combination of FIFO + LRU (to distinguish sequential scans from hot pages like the Parquet metadata).
7.
by ankoh 5y ago
Yes we do! DuckDB-Wasm can read files using HTTP range requests very similar to the sql.js-httpvfs from phiresky. The blog post contains a few examples how this can be used, for example, to partially query Parquet files over the network. E.
8.
by ankoh 5y ago
DuckDB-wasm is targeting the browser so it's not directly competing with Pandas (that's the job of native DuckDB). It's targeting use cases where you want to push analytical computation away from servers into the client (brow
9.
by ankoh 5y ago
The WebAssembly module is 1.6 - 1.8 MB brotli-compressed depending on the Wasm feature set. We're currently investigating ways to reduce this to around 1 MB. We further use streaming instantiation which means that the WebAssembly modul
10.
DuckDB-Wasm: Efficient analytical SQL in the browser(duckdb.org)
213 points by ankoh 5y ago|58 comments