ignoreusernames

6 ms·

1.
by ignoreusernames 2mo ago
> Heresy I suppose, but doesn't feel (to me) like it "specifically" has to be cons cells, as long as it's a "list" of some sort Yeah, "cons" is definitely a implementation detail. Maybe the central
2.
by ignoreusernames 2mo ago
> Homoiconicity, as I understand, is that the code is structured data that is easy to programmatically modify I think a more accurate description is that lisp code is just cons cells and cons cells is both how we write the code and how t
3.
by ignoreusernames 3mo ago
If safe keeper exposes the changes to the tables somehow, a type2 scd is just a windowed lag over the primary key sorted by the timestamp
4.
by ignoreusernames 3mo ago
Even when the layout is friendly to simd, auto vectorization can be finicky. As a programmer, it’s really annoying to be constantly inspecting compiler output to see if the code was properly vectorized. Even if it was, slight changes or com
5.
by ignoreusernames 3mo ago
Yeah, especially a bloomfilter which has a pretty easy formula for its false positive rate.
6.
by ignoreusernames 4mo ago
Don’t you think that the provider of the LLM is also a dimension on these discussions about responsibility? We often talk about the tech itself (LLM driven development) but how we access it is just as important imo. It’s either locked behin
7.
by ignoreusernames 4mo ago
I think this argument only holds if you believe that LLMs are at a point where it can handle any combination of craziness that you throw at it. From my own experience working with agents is that there’s “snowball of shit” effect. Small litt
8.
by ignoreusernames 5mo ago
Can you elaborate on this please? Do you mean that’s basically impossible for rust std to provide a default runtime that makes “everyone” (embedded on one end and web on the other) happy?
9.
by ignoreusernames 5mo ago
As of now I don’t think there’s an alternative. I’m not a Rust expert but the core issue to me is that “async” goes beyond just having a Futures scheduler. Async stuff usually needs network, disk, os interaction, future utilities(spawn) and
10.
by ignoreusernames 5mo ago
I may have missed something, but how does “sans-io” deal with CPU heavy code? For example, if there’s some heavy decoding/encoding required on the data? Does the event loop only drive the network side and the heavy part is done after t
11.
by ignoreusernames 5mo ago
Agree with the other commenters that the title is a bit too dramatic. The content was well written and got the point across. I still don’t have enough experience to have a strong opinion on Rust async, but some things did standout. On the g
12.
by ignoreusernames 1y ago
Another strategy to avoid redistribution is simply having a big enough number of partitions and assign ranges instead of single partitions. A bit more complex on the coordination side but works well in other domains (distributed processing
13.
by ignoreusernames 1y ago
> The reference implementation for Parquet is a gigantic Java library. I'm unconvinced this is a good idea. I haven't though much about it, but I believe the ideal reference implementation would be a highly optimized "serv
14.
by ignoreusernames 1y ago
> Folks I know in the industry are not very happy with databricks Yeah, big companies globing up everything does not lead to a healthy ecosystem. Congrats on the founders for their the acquisition but everyone else loses with movements l
15.
by ignoreusernames 1y ago
I recommend anyone who works with databases to write a simple engine. It's a lot simpler than you may think and it's a great exercise. If using python, sqlglot ( https://github.com/tobymao/sqlglot ) let's
16.
by ignoreusernames 1y ago
Same thing with columnar/vectorized execution. It has been known for a long time that's the "correct" way to process data for olap workflows, but only became "mainstream" in the last few years(mostly due to arr
17.
by ignoreusernames 2y ago
> Spark is "in-memory" in the sense that it isn't forced to spill results to disk between operations I see your point, but that's only true within a single stage. Any operator that requires partitioning (groupBys and
18.
by ignoreusernames 2y ago
just out of curiosity, why do you say that spark is "in-memory"? I see a lot people claiming that, including several that I've interviewed in the past few years but that's not very accurate(at least in the default case).
19.
by ignoreusernames 2y ago
> Most Parquet files are bulk exports from various data analysis pipelines or databases, often appearing as full snapshots rather than incremental updates I'm not really familiar of how datasets are managed by them, but all of the t
20.
by ignoreusernames 2y ago
From the announcement “As of now, we have mined 1,580 PySpark tests from the Spark codebase, among which 838 (53.0%) are successful on Sail. We have also mined 2,230 Spark SQL statements or expressions, among which 1,396 (62.6%) can be pars
21.
by ignoreusernames 2y ago
This is a fair argument that's often brought up but I never see actual raw data backing it up. Housing is fucked in several places around the world, including a bunch of countries in Europe that don't have any tax breaks for speci
22.
by ignoreusernames 2y ago
Early Athena (managed prestodb by AWS) had a similar bug when measuring colunar file scans. If it touched the file, it considered the whole file instead of just the column chunks read. If I’m not mistaken, this was a bug on presto itself, b
23.
by ignoreusernames 2y ago
Great series. If I'm not mistaken, there's an additional layer to the unreliable narrator part because the book is supposed to be a translation of that biography. So, when certain words are used, the reader knows that they don&#x2
24.
by ignoreusernames 2y ago
100% agree. mapReduce hype always seemed strange to me because it's basically the volcano paper from the 90s but with custom user defined operators instead of pre baked ones in a more traditional engine. To make everything worse, hado
25.
by ignoreusernames 3y ago
You can use a regular common lisp distribution like SBCL and just stick to foreign types https://www.sbcl.org/manual/#Foreign-Types this way you still have a GC for lighter tasks but with the option of manual memory ma
26.
by ignoreusernames 5y ago
Lisp is awesome, but I wish more places used regional pricing for online purchases. It's kinda crazy paying 40 dollars for an ebook since it's almost 1/5 of our minimum monthly wage (Brazil)