fovc

6 ms·

1.
by fovc 23d ago
> I would always pick an incremental batch load with a cursor value over a CDC connection. What's the benefit vs. something like Postgres's logical replication for CDC? IMO, the hard part of CDC is maintaining consistency in th
2.
by fovc 10mo ago
Łukasz Kaiser basically confirmed it in a podcast: https://youtu.be/3K-R4yVjJfU?si=JdVyYOlxUbEcvEEo&t=2624 > Q: Are the releases aligned with pre-training efforts? > A: There used to be a time not that long ago, m
3.
by fovc 10mo ago
> I also think it’s important to notice that a lot of these challenges they happen with humans too. The concept of prompt injection isn’t that different from social engineering, right? When somebody calls in and says, “Oh, I forgot my pa
4.
Why Building Agents Is Hard(artificialinvestment.substack.com)
3 points by fovc 10mo ago|1 comments
5.
by fovc 1y ago
Crossover from the other front page article. I tested out ChatGPT5 search mode and there are some good sources! https://chatgpt.com/s/t_68bd82908c0c8191b142b860ff91c9dc
6.
by fovc 1y ago
Very nice! For other readers, vc is short for verification condition and wp is short for weakest precondition.
7.
by fovc 2y ago
I wonder if the error propagation problem could be solved with a “branching” generator? Basically at every token you fork off N new streams, with some tree pruning policy to avoid exponential blowup. With a bit of bookkeeping you could make
8.
NoLiMa: Long-Context Evaluation Beyond Literal Matching(arxiv.org)
2 points by fovc 2y ago|0 comments
9.
by fovc 2y ago
Or see the explanation in video form here: https://m.youtube.com/watch?v=d0HJvGSWw8A Mamba has been discussed a lot here, and this seems like a promising line of inquiry for improvement
10.
DeltaNet Explained(sustcsonglin.github.io)
2 points by fovc 2y ago|1 comments
11.
by fovc 2y ago
And also make sure that slot is a symbol in the correct package. Or do like Elisp and do without packages but then have a 16 character prefix
12.
by fovc 2y ago
Having the data structures is nice and all, but using them is kind of painful. They are certainly second class. Having to use accessor functions or destructuring macros instead of just a period or -> is often annoying too. The lack of sy
13.
by fovc 2y ago
Sparse attention essentially combines 3 types of attention optimizations: 1. Compression of the query input vectors to reduce the size of the KV cache 2. Selectively computing uncompressed attention on a subset of tokens based on the compre
14.
Mamba-Shedder: Post-Transformer Compression for Efficient SSMs(arxiv.org)
1 points by fovc 2y ago|0 comments
15.
Reflections on 'The Bitter Lesson' (2021)(cognitivemedium.com)
1 points by fovc 2y ago|0 comments
16.
by fovc 2y ago
Ah thanks for the clarification! Do you happen to know if Nile/Gezira went anywhere?
17.
by fovc 2y ago
Great to see this is alive and progressing! I believe Ohm started life in Alan Kay’s research group, to build a graphical OS and office suite in 10k lines of code. I found this talk immensely inspiring https://m.youtube.com/
18.
Theoretical limitations of multi-layer Transformer(arxiv.org)
107 points by fovc 2y ago|22 comments
19.
by fovc 2y ago
> I feel like I'm taking crazy pills when I read about others' experiences. Surely I am not alone? You're not alone :-) I asked a very similar question about a month ago: https://news.ycombinator.com/item?i
20.
by fovc 2y ago
The “pair programming” approach with good models is just slow enough that I lose focus on each step. The faster models I’ve tried are not good enough except for straightforward things where it’s faster to just use emacs/LSP refactoring
21.
by fovc 2y ago
I still don’t understand how people are getting value out of AI coders. I’ve tried really hard and the commits produced are just a step up from garbage. Writing code from scratch is generally decent. But after a few rounds of edits the assi
22.
by fovc 2y ago
Makes a ton of sense! Is this for completions, patches, or new files?
23.
by fovc 2y ago
Maybe this? https://www.cis.upenn.edu/~bcpierce/tapl/
24.
by fovc 2y ago
Agree it’s not a toy. AWS implemented a large chunk of IAM in Dafny. Though IIRC they have their own non-public compiler to Java
25.
by fovc 2y ago
It’s also useful in typed languages to introduce an existentially quantified type
26.
by fovc 2y ago
Plants
27.
by fovc 2y ago
The requirement for having two copies of the table simultaneously on systems that make it easy to add but not subtract storage. Otherwise pg_repack has worked really well. We solved the 2x storage with partitions, but it feels like the tail
28.
by fovc 2y ago
Yes I know the slotted attribute is not in a __dict__, which definitely helps memory usage. But my point is that if the parent structure is itself in a dict, that access will swamp the L1 cache miss in terms of latency. Even the interpretat
29.
by fovc 2y ago
POSIX is weird, but NodeJS streams are designed to be misused
30.
by fovc 2y ago
It’s Python. Does L1 matter at all? I assume anything you’re accessing is behind a few pointers and __dict__ accesses anyway. For me it’s mostly about .attribute being more in line with the rest of the language. Kwargs aside, I find overus

More ›