dmoura

9 ms·

1.
Open Dataset: Vehicle Accidents(huggingface.co)
2 points by dmoura 2y ago|0 comments
2.
Nexar Dashcam Crash Prediction Challenge(kaggle.com)
10 points by dmoura 2y ago|4 comments
3.
Limitations of 'Understanding the Limitations of Mathematical Reasoning in LLMs'(desirivanova.com)
1 points by dmoura 2y ago|0 comments
4.
by dmoura 3y ago
Thank you for sharing your learnings and for your transparency! Congrats!
5.
by dmoura 3y ago
I prefer a SQL-like format. It’s not as complete but it cover most of the day-to-day use cases. Take a look at https://github.com/dcmoura/spyql (I am the author). Congrats on fq!
6.
The Five Dimensions of Sustainable Software Engineering(luiscruz.github.io)
2 points by dmoura 4y ago|0 comments
7.
Kangas: Explore Multimedia Datasets at Scale(github.com)
9 points by dmoura 4y ago|2 comments
8.
Neural Geometry and Rendering ECCV2022(ngr-co3d.github.io)
2 points by dmoura 4y ago|0 comments
9.
Reconstructing Training Data from Trained Neural Networks(giladude1.github.io)
11 points by dmoura 4y ago|0 comments
10.
by dmoura 4y ago
DuckDB is great! I love what you guys are building. The main gap for me is native support of JSON (lines), like you have for CSV and Parquet.
11.
by dmoura 4y ago
updated, thank you
12.
by dmoura 4y ago
Things you can do with SPyQL CLI that you can't with clickhouse local (AFAIK, top of my mind, not exhaustive): - use python code in your queries - import python libs (just install them with pip/conda) - write your one UDFs in Pyth
13.
by dmoura 4y ago
Author of the benchmark and of SPyQL here. ClickHouse is fantastic. Amazing performance. SPyQL is built on top of Python but still can be faster than jq and several other tools as shown in the benchmark. SPyQL can handle large datasets but
14.
by dmoura 4y ago
This is great! I am the author of SPyQL [1]. Combining JC with SPyQL you can easily query the json output and run python commands on top of it from the command-line :-) You can do aggregations and so forth in a much simpler and intuitive wa
15.
Command-line data analytics(danielcmoura.com)
103 points by dmoura 4y ago|25 comments
16.
SPyQL – SQL Powered by Python(spyql.readthedocs.io)
3 points by dmoura 4y ago|0 comments
17.
Make beautiful visualisations of large graphs online (2015)(github.com)
2 points by dmoura 4y ago|0 comments
18.
by dmoura 4y ago
Thank you all for your feedback. The benchmark was updated and the fastest tool is NOT written in Python. Here are the highlights: * Added ClickHouse (written in C++) to the benchmark: I was unaware that the clickhouse-local tool would ha
19.
by dmoura 4y ago
Are you able to calculate aggregates, like an average?
20.
by dmoura 4y ago
The initial idea was to focus on cmd line tools... I added pandas for comparison, as it is one of the most used libs to work with datasets. I will either remove Pandas from the equation or add Polars. By the way, I run some benchmarks and p
21.
by dmoura 4y ago
Yes, it is much faster, I will be updating the benchmark and reposting
22.
by dmoura 4y ago
Thank you!! :-D
23.
by dmoura 4y ago
Thank you Eaton! The truth is, if I was seeking to have the best performant tool I would not choose Python to start with... so performance is only a part of tools like the ones we are writing offer. Thank you for you tip, I will look into
24.
by dmoura 4y ago
Thank you!!
25.
by dmoura 4y ago
Yes, I agree with you. I guess the title is a bit too provocative... still trying to understand what tools I might have missed, and this seemed a good way of doing that... I do not want to give wrong impressions to people that only read the
26.
by dmoura 4y ago
Wow, I will check it out, thanks!!
27.
by dmoura 4y ago
Thanks @cube2222! Great! Sorry for overlooking that option. I will definitely add that option to octosql to make the comparison fair. Should I add it to the 3 queries?
28.
by dmoura 4y ago
Please take this claim and these results with a pinch of salt. spyql was not created with the goal of being the fastest tool for querying data, and it might be the case that the same tools with different datasets or in different use-cases o
29.
by dmoura 4y ago
Loved the simplicity of your solution :-) Congrats!
30.
by dmoura 4y ago
Thank you for your feedback! I understand your point of view, let me share mine. spyql is 100% Python code and it is not a thin layer over something else. Every row of data goes through a query engine built in python that takes care of eval

More ›