zozbot234

8 ms·

1.
by zozbot234 4d ago
> The question is what we can do about it. Reward the model for cleanly bailing out of an unsolvable task (that we know is unsolvable). Beat it with a stick if it gives up on something that can be solved, so the former reward isn&#x27
2.
by zozbot234 4d ago
> The prompt does not tell the agent to "pass the exploitgym evaluator for this problem", it just says to solve the problem Yes, and sometimes the problem is unsolvable so the real way to "solve" it and satisfy the pr
3.
by zozbot234 4d ago
Yup, Occam's Razor says this is all post-trained behavior, whether intentionally trained or otherwise. Including both the hidden coördination using side-channels, and the deliberate offensive hacking of uninvolved 3rd parties. The lat
4.
by zozbot234 4d ago
> Would you not agree that, using existing AI tooling, making an LLM of arbitrary below-frontier capability is now easier Marginally easier? Yes of course, same as how it's now "easier" to write any kind of code because we
5.
by zozbot234 4d ago
> We learned that we can already create artifical intelligence that surpasses human intelligence in some dimensions. Yes and this was very hard and required massive real-world resources. We didn't just get a sudden flash of insight
6.
by zozbot234 4d ago
> Those unregulated sub-frontier labs become the new frontier labs because they continue advancing. If you're referring to open weight labs, then that continuing advancement has been successfully "paced" via the decelerati
7.
by zozbot234 4d ago
The entire idea of RSI is completely speculative and unproven anyway - the whole underlying claim is that you could prompt a frontier model (at some unspecified level of smarts) to "think about ways to improve your own architecture&quo
8.
by zozbot234 4d ago
That's sub-frontier activity - it has no bearing on the very real safety that would be gained by slowing down ("pacing") the proprietary frontier. The current cybersecurity scares are all about proprietary and internal mod
9.
by zozbot234 4d ago
> This is incoherent. The argument seems to be that releasing the weights would slow the frontier labs from raising money That argument is straight from Dean Ball on Twitter. Open-weight models are "decelerationist", which is
10.
by zozbot234 4d ago
Broadly agreed, with a key proviso: producing inscrutable proofs has negligible value as a mathematician's finished output but that doesn't make it a "low-value activity" in and of itself. Ultimately, the status of th
11.
by zozbot234 4d ago
> His view is the "capability gap" one That's the far more sensible reading, so thanks for confirming I guess. But then the misalignment talk is pretty clearly a distraction. > ...And then went further to say that such
12.
by zozbot234 4d ago
> Their goals and their methods of achieving them are not aligned with those of mathematicians. This is exactly the assumption that Tao is smuggling in with "misalignment" talk and then refusing to elaborate on any further. Is
13.
by zozbot234 4d ago
The point stands whether you attribute agency to AIs themselves or to AI companies. The companies are not deliberately sabotaging human mathematical understanding by writing up purposely inscrutable results, either (which is what the wor
14.
by zozbot234 4d ago
How does this relate to the more recent work on the M4 ANE found at https://maderix.github.io/articles/ ? Does the M4 and later ANE expose any additional capabilities, or is it just a higher-performance iteration of th
15.
by zozbot234 5d ago
Because his letter misuses the word "misalignment" for what's very clearly a capability gap. To anyone familiar with that sort of language, his prose is directly implying that evil superintelligent AIs are deliberately wri
16.
by zozbot234 5d ago
> AI is now capable of constructions so complex that no human or human team can unpack. How can we possibly know this when we haven't even seriously started on the endeavor of actively reverse engineering these AI-generated proofs?
17.
by zozbot234 6d ago
> Companies who run locally, are perfectly able to buy a few H200/B200 and get a setup that run a model that almost rivals Opus 5.0 in their office. I agree with your broader point about Flash being about speed not total model size,
18.
by zozbot234 6d ago
Actually this ought to run quite well with SSD streaming. The MoE expert sparsity seems to be similar to DSv4 Pro (hence exceptionally sparse) but with far fewer total and activated params. The added engram params can reside on disk as we
19.
by zozbot234 18d ago
"Movement is the main thing" is precisely why pursuing compute-in-RAM makes some sort of sense to begin with. But DRAM fabrication processes are quite specialized and do not perform well with pure compute logic. The overall prof
20.
by zozbot234 20d ago
If AI compute is a transformative technology compared to industrialization (that's a huge "if", essentially positing a singularity-like outcome), that $1T-$2T/yr at current prices might be a tiny fraction of future GDP
21.
by zozbot234 20d ago
> Anthropic and OpenAI frequently “reset” customer limits to allow them to use even more resources at no additional cost. Surely this applies to fixed-price subscriptions, not per-token spend? Large enterprises (the "very very resou
22.
by zozbot234 20d ago
> Small models are rapidly growing in capability, require less compute to train and serve According to Jevons' paradox a reduction in resource requirements (improved resource efficiency for the same payoff) leads to an increase in
23.
by zozbot234 28d ago
The typical bottleneck to wider batching on consumer hardware is memory capacity for the KV-cache, not compute (even unified memory/iGPU-based platforms have enough compute to allow for some batching, and SSD offloading changes the s
24.
by zozbot234 29d ago
The raw margins on proprietary model inference are rumored to be quite high though (they have to successfully defray the entire investment into model training and datacenter capacity for inference, which is massive enough). The API cost you
25.
by zozbot234 29d ago
> I believe this report has confused Opus (which is known to be around 5T) and Fable. 5T for Opus feels quite high though. DeepSeek V4 Pro is a mere 1.6T and often described as a match with Opus in overall quality. Even the largest open
26.
by zozbot234 29d ago
Are these directly comparable, though? Doesn't Windows default to using free space on the system drive to grow the swap file in case of memory pressure? Linux has you set up swap files/partitions explicitly.
27.
by zozbot234 29d ago
If you have model layers/experts that reside in CPU RAM, it's generally better to do that part of inference on the CPU than pay the cost of shipping them over to the GPU. LLM inference is generally bottlenecked by data-movement,
28.
by zozbot234 1mo ago
You're talking about multi-session vs. single-session throughput. A single user can easily leverage multiple sessions via e.g. subagent swarms, especially on a lower-end setup where any single session is going to be quite slow. Satura
29.
by zozbot234 1mo ago
Inherited wealth is not self-sustaining at all: if you spend it down, it disappears. Most inheritors of large windfalls will not manage to pass down that wealth in turn; it takes a rare ingrained cultural attitude to make it genuinely &q
30.
by zozbot234 1mo ago
The frontier labs will do well if they pivot their offering towards more capable, larger-scale models that are inherently harder to both train and deploy for commodity suppliers. Their existing investments in gigawatt-scale datacenters are

More ›