kherud

6 ms·

1.
by kherud 1mo ago
So far I haven't seen a single model succeeding at transcribing sheet music, but I just tested it again with 5.6 Sol and it nailed the small test case. Fluently reading music requires multiple years of training for most people, but I f
2.
by kherud 2mo ago
Imagine what amazing SVG generators we could have if Simon had randomized the target image from the start (and companies wouldn't just overfit on pelicans).
3.
by kherud 4mo ago
Feedback about creatine often seems mixed. Many love it, but many also report problems. Just in this thread there are people talking about heart palpitations and sleep problems. I have the theory that these side effects and mixed experience
4.
by kherud 5mo ago
LLMs are extremely capable at problem solving. Presumably because you can autonomously learn a lot of it. But can you somehow account for things like long-term maintainability and code quality (whatever that means) or do you always have to
5.
by kherud 5mo ago
If I'd have to make one recommendation it's David August's Boiler Room set [1]. It has such a coherent flow through the whole set, it makes me fly through multiple hours if not days of work. [1] https://www.youtube
6.
by kherud 6mo ago
SQLite seems very powerful for building FTS (user enters free text, expects high precision/recall results). Still, I feel like it's non-trivial to get good search quality. I think the naive approach is to tokenize the input and ap
7.
by kherud 9mo ago
One interesting detail: In previous years, Joscha Bach gave a talk on AI, consciousness, and related topics (see e.g. [0]). A similar talk was planned for this year as well, but after emails between him and Epstein were made public (see his
8.
by kherud 1y ago
I'm a fan of antlr-ng. It's a solid upgrade if you're already using antlr. In my experience, they're fully compatible. antlr's ALL(*) parsing is relatively powerful for a parser generator, but it lacks support for i
9.
by kherud 1y ago
It's probably "Reflections on Palantir" https://news.ycombinator.com/item?id=41855006
10.
by kherud 1y ago
Is there a general solution to this problem? I assume you can only start buffering tokens once you see a construct, for which there are continuations, that once completed, would lead to the previous text being rendered differently. Of cours
11.
by kherud 2y ago
That was my association as well! Dune even uses similar vocabulary. For example someone mentioned "pranayama" in this thread, which sounds a lot like Dune's "Prana-bindu". Really makes me wonder about Frank Herbert&
12.
by kherud 2y ago
Aren't LLMs much more limited on the amount of output tokens than input tokens? For example, GPT-4o seems to support only up to 16 K output tokens. I'm not completely sure what the reason is, but I wonder how that interacts with C
13.
by kherud 2y ago
Maybe you're already aware of it, but there is difftastic [0], which is a syntax aware diff tool that can also be used with git. Its understanding of syntax is based on treesitter, so it works for most languages. Although I haven'
14.
by kherud 2y ago
Interesting, thanks for sharing! Do you have an explanation or idea why compilation slows some architectures down?
15.
by kherud 2y ago
I think it's more about invested work vs. reward. Mindless browsing is one of the lowest work activities, but the influx of information is highly rewarding for the brain. That's why it's so addicting. Programming and OS insta
16.
by kherud 2y ago
Let's say you want to show a modal, which fetches some data and modifies the state. Based on this, new children are rendered which again fetch state. The problem of "spaghetti fetching" becomes worse the more levels of recurs
17.
Does GPT-4o use OCR for vision?(kherud.github.io)
1 points by kherud 2y ago|0 comments
18.
by kherud 2y ago
Shouldn't this theory be testable? The response time for an image of the same size should remain constant (assuming a generated response of constant size). You could then try to put an increasing amount of text inside of the image. If
19.
by kherud 2y ago
I think this comment explains it https://github.com/ggerganov/llama.cpp/discussions/4130#disc... As far as I understand (and mcharytoniuk should better confirm this), llama.cpp allows to chunk the context win
20.
by kherud 2y ago
Now that context length seems abundant for most tasks, I'm wondering why sub-word tokens are still used. I'm really curious how character-based LLMs would compare. With 2 M context, the compute bottleneck fades away. I'm not
21.
by kherud 3y ago
Thank you for sharing! On a tangent: I'm wondering if there are any good open source models/libraries to reconstruct audio quality. I'm thinking about an end-to-end open source alternative to something like Adobe Podcast [1]
22.
by kherud 3y ago
Why is this the expected result? The original transformer algorithm has a n^2 computational complexity, where n is the amount of tokens. As far as I know, there are some improvements which bring it down to something like n*log(n). A linear
23.
by kherud 3y ago
Can somebody please explain how quantization below 8 bit works? Since a byte is the smallest addressable unit I think, is the dimensionality of the weights somehow reduced?
24.
by kherud 3y ago
Answer Set Programming is an incredibly powerful tool to declaratively solve combinatorial problems. Clingo is one of the best open source implementations in my opinion: https://github.com/potassco/clingo - Based o
25.
by kherud 4y ago
I can recommend the puzzle books by Raymond Smullyan [0], for example "Satan, Cantor and Infinity". Among others, this contains puzzles about infinities and apparent paradoxes associated with them. Reading is mostly a form of con
26.
by kherud 4y ago
Without being particularly familiar with it, I wonder how BDDs differ from the generalization to multivalued decision diagrams (MDDs)? For example, in constraint satisfaction problems, variables often have more than two values. However, in