marsupialtail_2

6 ms·

1.
by marsupialtail_2 2y ago
Butterfly flies 2600 miles across the ocean just to be caught by a human in a jar to be DNA sequenced ...
2.
by marsupialtail_2 2y ago
The sincerest form of flattery is when AWS decides to come up with a big consortium to displace you with some open source. Incidentally the most effectively way to stall a project according to the CIA is to have a huge guiding committee wit
3.
by marsupialtail_2 3y ago
code: https://github.com/Vince7778/Emojile
4.
Wordle but with Emojis(vince7778.github.io)
1 points by marsupialtail_2 3y ago|1 comments
5.
by marsupialtail_2 3y ago
No just curious. I understand how your indexing structure based on SSTables could find it challenging to support substring search in general. I think it tradeoff between fast querying and flexible functionality
6.
by marsupialtail_2 3y ago
Glad this is getting some love. This is seriously good software. Have you guys supported generic substring search yet? I recall it was not supported as of a few months ago.
7.
by marsupialtail_2 3y ago
Thanks for the shoutout!
8.
by marsupialtail_2 3y ago
Yes but there is also the inverse carrot problem. E.g. if the pilots have radar, they are more liable to rely on it and neglect other aspects of flying. Similarly in business, it is simply harder for folks who grew up rich to develop the le
9.
by marsupialtail_2 3y ago
Hi Justin, you might be interested in my blog: https://github.com/marsupialtail/quokka/blob/master/blog/bac... advocating a cloud based approach. You don't have to use the system I am building,
10.
by marsupialtail_2 3y ago
Perhaps I charged too little when I contracted away my 10x random forest inference solution...
11.
by marsupialtail_2 3y ago
SQL support is very challenging. I work on Quokka ( https://github.com/marsupialtail/quokka ). I support Iceberg reads. Recently we are adding SQL support from just parsing the DuckDB logical plan, though that is very ch
12.
by marsupialtail_2 3y ago
While we are on this topic, the challenge with data lakes for Python based projects like Daft and Quokka (what I work on) is the poor Python support for data lakes like Delta, Iceberg and Hudi. Delta has the best support but its Python API
13.
by marsupialtail_2 3y ago
Hi -- I am the author of Quokka: https://github.com/marsupialtail/quokka , trying to be distributed Polars. I am trying to go for API compatibility, or at least supporting most of the API. I am not focused on complex da
14.
Open Vector Data Lakes(blog.lancedb.com)
2 points by marsupialtail_2 3y ago|0 comments
15.
Why your dataframe library needs to understand vector embeddings(github.com)
3 points by marsupialtail_2 3y ago|1 comments
16.
by marsupialtail_2 3y ago
Open for rebuttals from vector database vendors, especially this one: https://github.com/jdagdelen/hyperDB
17.
by marsupialtail_2 3y ago
You will always be limited by network throughput. Sure that wire is getting bigger but so is your data
18.
by marsupialtail_2 3y ago
In case people are interested, I wrote a post about fault tolerance strategies of data systems like Spark and Flink: https://github.com/marsupialtail/quokka/blob/master/blog/fau... The key differenc
19.
by marsupialtail_2 3y ago
I hope in a good way
20.
by marsupialtail_2 3y ago
Can you comment on main differences between this and Aurora?
21.
by marsupialtail_2 3y ago
It's more about the API -- currently the API mimics Polars. I actually use DuckDB for a lot of the computation.
22.
Show HN: Quokka -- Distributed Polars on Ray(github.com)
5 points by marsupialtail_2 3y ago|5 comments
23.
by marsupialtail_2 4y ago
would love to collaborate on an integration with pyquokka ( https://github.com/marsupialtail/quokka ) once I put out a stable release end of this month :-)
24.
by marsupialtail_2 4y ago
If you work on a project long enough eventually you forget about how parts of your project work, and this automatically happens
25.
Ask HN: How fast can you hit all 34M items in Hacker News Firebase API?
8 points by marsupialtail_2 4y ago|5 comments
26.
Distributed fault tolerance made simple(github.com)
2 points by marsupialtail_2 4y ago|0 comments
27.
Fault tolerance for distributed data systems is really quite simple(github.com)
7 points by marsupialtail_2 4y ago|0 comments
28.
by marsupialtail_2 4y ago
If you make everything open source...
29.
by marsupialtail_2 4y ago
me too. Trino for one would be a good start. Adding support for those data lakes is really hard though if you want good performance.
30.
by marsupialtail_2 4y ago
I think the blog post should point out very early that Onehouse is a Hudi company. There are some other recent benchmarks published in CIDR by Databricks that might paint a different picture: https://petereliaskraft.net/res&

More ›