6 ms·
Same here. But have you tried duckdb? You can do sql in the pandas dfs and it is fast af. https://duckdb.org/2021/05/14/sql-on-pandas.html https://duckdb.org/2
by agacera 3y ago
Same here.
But have you tried duckdb?
You can do sql in the pandas dfs and it is fast af.
https://duckdb.org/2021/05/14/sql-on-pandas.html https://duckdb.org/2021/05/14/sql-on-pandas.html
- cookieperson 3y agoDuckdb is sick. You can also do queries on parquet, etc.
- avnigo 3y agomydf = pd.DataFrame({'a' : [1, 2, 3]}) print(duckdb.query("SELECT SUM(a) FROM mydf").to_df()) I can see the appeal, but if you're working in Python, something doesn't sit right with me when having to write out variable names as strings. E.g., if I want to refactor the code, my LSP or parser won't pick up those references. > The SQL table name mydf is interpreted as the local Python variable mydf [...] Not only is this process painless, it is highly efficient. It might be painless and convenient at first, but I feel like this could get you in trouble down the line. Is there a way to avoid this?
- freilanzer 3y agohttps://duckdb.org/docs/guides/python/ibis.html https://duckdb.org/docs/guides/python/ibis.html