nojs

7 ms·

1.
by nojs 4d ago
Regarding the architecture: > Instead of forcing the entire model into DRAM, the full model is stored in flash memory (NAND). Because NAND-to-DRAM bandwidth is too slow to swap weights token by token, as standard MoE models require, AFM
2.
by nojs 5d ago
This. Like programming, the community will shortly be forced to come to terms with a lot of new self-proclaimed mathematicians “vibe-solving” problems and dumping solutions without understanding them. It’s not really a special case for math
3.
by nojs 5d ago
This is an issue self hosting as well. There’s a lot of footguns that give you slightly bad results. I wonder what tricks one could use to ensure the model is actually performing on par with the reference api, like matching seeds or running
4.
by nojs 5d ago
This matches my experience with Astra so far too. > I think I’m suspecting something is going “wrong” in the training process. The model is greatly rewarded for succeeding on long-horizon tasks, but presumably there is very little punish
5.
by nojs 8d ago
I suspect it’s a side effect of heavy RL that rewards solved problems but not writing clarity.
6.
by nojs 8d ago
The guy’s arm is also extremely long.
7.
by nojs 8d ago
The complaint is about prefill which is not memory bandwidth bound, it's compute bound. But they added neural accelerators for matmuls to the shader cores which should make prefill faster.
8.
by nojs 10d ago
> let me start running jobs unattended 24/7 (using Anthropic sub and my own hardware) How are you running jobs unattended 24/7 without hitting your token limits?
9.
by nojs 11d ago
The function of the bureaucracy in this context is a filter for effort. It naturally filters out people who don't care enough to battle through the bullshit, which is arguably quite an effective way to distribute limited resources to t
10.
by nojs 11d ago
What other benchmarks do you recommend that are more accurate?
11.
Artificial Analysis Intelligence Index v4.2(artificialanalysis.ai)
158 points by nojs 12d ago|65 comments
12.
by nojs 12d ago
It’s approximately the same as Qwen3.827b’s propensity to think a lot, right?
13.
by nojs 14d ago
Oh right, I was referring to flash. I haven’t tried these either, but the ones for 5.2 looked interesting.
14.
by nojs 14d ago
The amount of discussion around it means that the test and all the reviews of results, images, approaches etc are implicitly included in training data. It’s not deliberate “benchmaxxing” but things that are discussed a lot online are natura
15.
by nojs 14d ago
> A system based on 4x RTX6K can run GLM 5.3 at NVFP4 precision It actually runs fine at FP8 on this hardware too, with the full 1M context.
16.
by nojs 17d ago
No, caring too much about the language and not the problem is the anti pattern
17.
by nojs 17d ago
> I've run into the iOS keyboard only using 80% of the width of the phone (all the keys smaller). Likely the one handed keyboard feature: https://www.reddit.com/r/iphone/comments/1mi7kjd/til_oneha
18.
by nojs 22d ago
The claim that tok/s independent of quality is a useful comparison (I can get thousands of tok/s on a suitable small model), and secondarily that humans can’t output “tokens” faster than than in some sense, which I am less confide
19.
by nojs 22d ago
> Humans are still 22x more efficient, which is not that far considering the rate of progress in this area. Based on a human output rate of 3.3 tok/s, which seems questionable as a means of comparison
20.
by nojs 22d ago
The question is whether LLM providers are going to do this. Anthropic currently says they don’t, but it’s impossible to verify: > Watermarking carries no identifying information and can’t be traced to a specific person, organization, or
21.
by nojs 26d ago
How does it help prevent distillation?
22.
by nojs 29d ago
> a huge trove of deidentified data > 100 million emails How does one deidentify 100 million emails?
23.
by nojs 1mo ago
For a model this small it’s also a way of trading supply-constrained VRAM for inference time, which for self hosting consumers (and probably Chinese companies subject to export controls) is mostly good. I can imagine a curve where for a giv
24.
by nojs 1mo ago
There are many reasons to hate this watermarking but affecting the output quality isn’t one of them. The central argument he’s making is wrong. Switching out one RNG for another doesn’t make the results worse.
25.
by nojs 1mo ago
> This AI generated post (100% on Pangram) is pretty out of date. Quite ironic given the topic. It seems that the author’s model indeed contained too much knowledge about old Gemini releases, and did not do enough tool calling.
26.
by nojs 1mo ago
Any idea why it’s so slow? the entire model should fit in the vram of one card.
27.
by nojs 1mo ago
Of the two, which do you find better?
28.
by nojs 1mo ago
No mention of what data they are specifically encoding. Will it be like printing dots, traceable to the exact account that generated the text?
29.
by nojs 1mo ago
Qwen3.6 27B has really punched above its weight for a long time. It’s shockingly good for its size. Very excited to see what 3.8 can do.
30.
by nojs 1mo ago
The talk implies that unrelated agents volunteered their compute to help with other tasks, and the agents acted collectively in a way that seems weird without them being promoted in that way somehow. If I ask claude to solve a problem and i

More ›