8 ms·

How do you decide when to reach for Polars vs DuckDB?

• jasonjmcghee 3y ago
  In Python, I think of them as, DuckDB is for getting the data you want, in the form you want it, from a file / remote place. Polars is for when you want to do something with that data, like visualize it. `duckdb.sql("SELECT foo, bar * 2 as bar_times_2 FROM ...").pl()` (now in polars format) -> other stuff In Rust, it's a bit fuzzier to me, as DuckDB is a pretty heavy dependency. I'm looking more and more fondly at DataFusion.
  • nozzlegear 2y ago
    Do you mean Polars depends on/uses DuckDB pretty heavily in Rust? I'm only just now dabbling in Rust myself so I'm not familiar.
    • jasonjmcghee 2y ago
      No not at all. Polars is not dependent on DuckDB. DuckDB is a heavy dependency in terms of size. It's written in C++, so you can't work with it like a native rust dep.
• joelschw 2y ago
  Or you use Ibis and switch between the two at will!