wesm

7 ms·

1.
Ghosthub: Tmux/SSH-native macOS terminal based on libghostty(ghosthub.ai)
3 points by wesm 1mo ago|0 comments
2.
by wesm 4mo ago
See also https://wesmckinney.com/blog/mythical-agent-month/
3.
by wesm 7mo ago
See also The Mythical Agent-Month https://wesmckinney.com/blog/mythical-agent-month/
4.
The Mythical Agent-Month(wesmckinney.com)
5 points by wesm 7mo ago|0 comments
5.
by wesm 7mo ago
Super well! I don't work without this tool running in the background supervising all the agents' work
6.
by wesm 7mo ago
I've been building https://roborev.io/ (continuous background code review for agents) essentially as a cope to supervise the poor quality of the agents' work, since my agents write much more code than I can possib
7.
From Human Ergonomics to Agent Ergonomics(wesmckinney.com)
4 points by wesm 8mo ago|1 comments
8.
Roborev: Automated background code review for your agentic commits(github.com)
1 points by wesm 8mo ago|0 comments
9.
roborev: Background agent to review your Git commits with Codex or Claude Code(github.com)
4 points by wesm 8mo ago|0 comments
10.
Can LLMs give us AGI if they are bad at arithmetic?(wesmckinney.com)
1 points by wesm 10mo ago|1 comments
11.
Moneyflow: Power user terminal UI for personal finance data(moneyflow.dev)
3 points by wesm 11mo ago|0 comments
12.
by wesm 3y ago
I was especially excited to learn that RZ is built on Apache Arrow internally, which makes it easy to integrate with other Arrow-based applications and the emerging "Composable Data Stack". Really exciting stuff, they're just
13.
by wesm 4y ago
If you read my slide decks over the last 7 years or so (while I've been working actively on Arrow and sibling projects like Ibis) I've been saying exactly this. See e.g. https://ibis-project.org/
14.
by wesm 6y ago
Almost no database systems support multidimensional arrays. So they are not appropriate for many use cases? * BigQuery: no * Redshift: no * Spark SQL: no * Snowflake: no * Clickhouse: no * Dremio: no * Impala: no * Presto: no ... list conti
15.
by wesm 6y ago
I challenge you to have a closer look at the project. Deserialization by definition requires bytes or bits to be relocated from their position in the wire protocol to other data structures which are used for processing. Arrow does not requi
16.
by wesm 6y ago
> Arrow's serialization is Protobuf Incorrect. Only Arrow Flight embeds the Arrow wire format in a Protocol Buffer, but the Arrow protocol itself does not use Protobuf.
17.
by wesm 6y ago
There's no serde by design (aside from inspecting a tiny piece of metadata indicating the location of each constituent block of memory). So data processing algorithms execute directly against the Arrow wire format without any deseriali
18.
by wesm 6y ago
Microsoft is also on top of this with their Magpie project http://cidrdb.org/cidr2021/papers/cidr2021_paper08.pdf "A common, efficient serialized and wire format across data engines is a transformational deve
19.
by wesm 6y ago
(Wes here) I appreciate the Arrow shout-out but note that Apache Arrow has been a major open source community collaboration and not something I can take sole credit for.
20.
by wesm 6y ago
There is no “JIRA politics” blocking the LZ4 work, only a lack of volunteers to do the development and testing.
21.
by wesm 6y ago
Keep in mind that Arrow Java<->C++/Python interop has been in production use in Apache Spark and elsewhere for multiple years now. We have avoided some of the mistakes of past projects by really emphasizing protocol integration t
22.
by wesm 6y ago
Again, I have to object to your use of “arrow/parquet”. These are not the same open source projects and while people use them together it isn’t fair to the developers of each project for you to discuss them like a single project.
23.
by wesm 6y ago
See http://arrow.apache.org/faq/index.html#what-about-arrow-file... You can store them long-term if you want (and you'll still be able to read them 5 years from now) but we aren't optimizing the Arrow IPC for
24.
by wesm 6y ago
What you've written sounds like a criticism of the JVM data analytics ecosystem (the Java Parquet library in particular) and not Apache Arrow itself. Parquet for Java is an independent open source project and developer community. For e
25.
by wesm 6y ago
hi, Wes (Apache Arrow co-creator and Python pandas creator) here! If you're wondering what this project is all about, my JupyterCon keynote (18 min long) from 3 years ago is a good summary and the vision / scope for what we'v
26.
by wesm 6y ago
This isn't accurate -- there are multiple query engine subprojects within Apache Arrow.
27.
by wesm 6y ago
Arrow: * Standardizes binary interop and "serialization" of large structured data, removing all conversions / serialization at ingest and export boundaries. This alone can mean > 2-100x performance improvement in an applic
28.
by wesm 7y ago
I wasn't aware of it but will take a look. Thanks!
29.
by wesm 7y ago
To be clear for anyone reading, we're parsing and generating the data-related protobufs ourselves, and retaining ownership of the memory returned by gRPC to obtain zero copy. The C++ details are found in https://github.com&#
30.
by wesm 7y ago
No. If you want to talk about it come on the Apache Arrow mailing list

More ›