6 ms·
Happy to see activity around Polars. This has been my go-to library for data processing due to the enhanced ergonomics compared to Pandas and SQL. But they wer
by Kydlaw 14d ago
Happy to see activity around Polars. This has been my go-to library for data processing due to the enhanced ergonomics compared to Pandas and SQL.
But they were a bit quiet lately, and I started looking more and more into DuckDB recently… until the recent acquisition of DuckLab by AWS
- f311a 14d agoI've been using clickhouse-local for quite some time, instead of DuckDB. There is also chDB. After using pandas for 10 years, I favor SQL now, for some reason.
- mihaelm 14d agoMaybe because it's like a swiss army knife for data work, regardless of whether you need it for OLTP or OLAP workloads. Having different SQL dialects is a bit annoying, but the base is the same more or less, so switching doesn't come at too big of a cost.
- efromvt 14d agoAs a huge duckdb fan, I'd love to see chDB to get proper windows support - that would make it real competition (having WASM coverage is already a big step) which would be good for the space as a whole.
- Kydlaw 14d agoI haven't tried chDB yet, but I heard about it. Thank you for reminding me of that option. I use SQL in data pipelines and processing that is going to require interoperability. But for data exploration, I usually prefer Polars (imo it is easier to work with text, semi-structured data, etc.)
- geysersam 14d agoInteresting, why do you typically prefer clickhouse local to duckdb?
- f311a 14d agoBecause we use regular CH in production. I'm pretty familiar with it and I can always run large queries on a production server with 1TB of RAM. Regular CH also support external data sources, so I can read 500GB of JSON from S3 and group by it on production server very fast and in memory.