9 ms·
Nothing against what I said in there. Distillation is far from enough to get to the frontier. Its at best a ramp (the most efficient one used by everyone ) that
by retinaros 23d ago
Nothing against what I said in there. Distillation is far from enough to get to the frontier. Its at best a ramp (the most efficient one used by everyone ) that shortcut and saves millions of rl runs before a model moves.
- dbbk 23d agoYou said there isn't evidence of distillation. Yes there is
- retinaros 22d agoI said there is no evidence distillation is all you need as per the initial comment "The fact that AI models can be so easily distilled and replicated is such a stroke of luck." it is far from trivial to reproduce the capabilities of anthropic models for instance
- wolvoleo 23d agoAnd probably saves a lot of energy which would be good for the planet
- JacobAsmuth 23d ago"No evidence" -> Large lab saying that they use distillation when training their models
- retinaros 22d agothe initial post infers that distillation is all you need. it is not. in 2026 you need large scale distributed inference of gigantic models, rl envs and millions of dollars runs to get to something decent. if you think glm just has to sft on traces of claude to edge Mythos on some cyber benchmark you are fooled.