Vetch

8 ms·

1.
by Vetch 8d ago
The most charitable explanation I can think of for this is something like regression to the mean. When a model is first released, there'll be a subset of users who, just by chance, sample the highest quality band of the distribution th
2.
by Vetch 20d ago
This work feels more like The Truth Mines in Diaspora. Permutation city seems relevant only if you think LLMs are hosts to minds.
3.
by Vetch 24d ago
The weights + the architecture are already 100% of the code, the transformer is just a mathematical expression + helper programs whose sources are provided. The transformer itself is not even a stateful program, so a it is no more a binary
4.
by Vetch 3mo ago
What constitutes serious work and how seriously have you tried to do serious work with them? While those trying to claim a 30B dense model can match Opus 4.6 are engaging in either beyond over-excessive over-exaggeration or performing rathe
5.
by Vetch 3mo ago
Interesting thought to consider in principle but fails because gorilla brains continued to evolve too, just along a different path. They're not snapshots of ancestral species locked in time.
6.
by Vetch 6mo ago
This sounds very like Licklider's essay on Intelligence Amplification: Man Computer Symbiosis, from 1960: > Men will set the goals and supply the motivations, of course, at least in the early years. They will formulate hypotheses. T
7.
by Vetch 6mo ago
The relu/if-then-else is in fact centrally important as it enables computations with complex control flow (or more exactly, conditional signal flow or gating) schemes (particularly as you add more layers).
8.
by Vetch 6mo ago
This is essentially what any relu based neural network approximately looks like (smoother variants have replaced the original ramp function). AI, even LLMs, essentially reduce to a bunch of code like let v0 = 0 let v1 = 0.40978399
9.
by Vetch 8mo ago
I'm not sure that's the fully right mental model to use. They're not searching randomly with unbounded compute nor selecting from arbitrary strategies in this example. They are both using LLMs and likely the same ones, so wil
10.
by Vetch 11mo ago
Then the point still stands, this makes things even worse given that it's adding its own hallucinations on top, instead of simply relaying the content or idealistically, identifying issues in the reporting.
11.
by Vetch 11mo ago
Being tall doesn't automatically make you good or dominant at basketball, you can even be too tall. Wemby might just be at that threshold, but the unusual thing about him is his dexterity despite his height; such maneuverability and fl
12.
by Vetch 1y ago
It's an artifact of post-training approach. Models like kimi k2 and gpt-oss do not utter such phrases and are quite happy to start sentences with "No" or something to the tune of "Wrong". Diffusion also won't h
13.
by Vetch 1y ago
You are right and the idea of LLMs as lossy compression has lots of problems in general (LLMs are a statistical model, a function approximating the data generating process). Compression artifacts (which are deterministic distortions in reco
14.
by Vetch 1y ago
Unless you're also writing your own graphics and game engine from scratch, if you're making a truly novel and balanced game, then it should not be possible to crank out code with AI. When working in engines, the bulk of the work i
15.
by Vetch 1y ago
Why would the proportion of high quality games increase? The number yes, but I expect not the proportion. Lowering the entry barrier means more people who have spent less time honing their skills can release something that's lacking in
16.
by Vetch 1y ago
But why isn't this merely papering over a more fundamental issue with how these models are "aligned"? LLMs are, for example, not inherently sycophantic. kimi k2 and o3 are not, and Sydney, mentioned in the blog post, was most
17.
by Vetch 1y ago
The brain is certainly vastly more energy efficient at inference than LLMs on GPUs. But it looks like you're trying to make a different argument, that an LLM can spend less energy than a human to complete a given task. Unfortunately, y
18.
by Vetch 1y ago
That math is for random projections? Note that JL lemma is a worst case guarantee and in practice, there's a lot more distortion tolerance than the given bounds would suggest. Concepts tend to live in a space of much lower intrinsic di
19.
by Vetch 1y ago
Compounding with learn and iterate, humans also build abstractions which significantly shorten the number of steps required. These are more expressive programming languages, compilers and toolchains. We also build engines, libraries, DSLs a
20.
by Vetch 1y ago
> he's inability to see its application to modern compute held the field back by years. I find Schmidhuber's claim on GANs to be tenuous at best, but his claim to have anticipated modern LLMs is very strong, especially if we ar
21.
by Vetch 1y ago
The non-o-series models from OpenAI and non-Opus (although I have not tried the latest, so it's possible that it too joins them) from Anthropic are cloyingly sycophantic, with every other sentence of yours containing a brilliant and fa
22.
by Vetch 1y ago
This contains a common misstep (or misgeneralization of an analogy) among those who are much more familiar with computers than with the brain. The brain is not digital and concepts like frames per second and resolution don't make much
23.
by Vetch 1y ago
I think the point remains that few have been able to catch up to OpenAI. For a while it was just Anthropic. Then Google after failing a bunch of times. So, if we relax this to LLMs not by OpenAI, Anthropic or Google, then Deepseek is really
24.
by Vetch 1y ago
There is skill to it but that's certainly not the only relevant variable involved. Other important factors are: Language: Syntax errors rise, and a common form is the syntax of a more common language bleeding through. Domain: Less so t
25.
by Vetch 1y ago
LeBron is one of the rare individuals at that intersection of high athleticism and mental capability. It's why at the age of 40, well past his athletic prime, he's still a top NBA player. He has Magnus-level chunking ability ena
26.
by Vetch 2y ago
The essence of the article is that self-correction exists as a nascent ability in base models already (more robustly in some like Qwen than others). This is highly reminiscent of Chain of Thought, which was found to be a capability already
27.
by Vetch 2y ago
I'm also ashamed to say I've also never seen any of his movies and TV series but this still hits hard because of his influence on some my most cherished fictional properties. These are Alan Wake/Control, Silent Hill 1&2,
28.
by Vetch 2y ago
This is an interesting proposition. Have you tested this with the best open LLMs?
29.
by Vetch 2y ago
This seems wrong. Reasoning scales all the way up to the discovery of quaternions and general relativity, often requiring divergent thinking. Reasoning has a core aspect of maintaining uncertainty for better exploration and being able to te
30.
by Vetch 2y ago
The information on the creative step which you provided to o1, was also the key step and contained almost all the difficulty. The hope is that 2025 models could eventually come up with solutions like this given enough time, but this is also

More ›