6 ms·
What actually is "scaling post-training"?
by kleiba2 1mo ago
What actually is "scaling post-training"?
- FergusArgyll 1mo agoMore RLVR. Give it verifiable problems, if it doesn't find a solution move on, if it does, use that as a reward signal.
- Gecko4072 1mo agoCan’t this be extended quite far? Use a cerebras-served model, use verification techniques to generate and solve millions of problems and then use that as training?
- gvkhna 1mo agoThat’s the whole point, just cost and compute limitations in your way (mostly).
- kevincox 1mo agoThis isn't latency bound, it is trivially parallelize. So you want to run it on the most efficient compute you have, not the fastest.
- andai 1mo agoOpenAI was bragging the other day about how HLE on Cerebras only took 11 hours. But you're saying if they just spun up a separate cheapo VM for each task, it would have finished faster and cheaper?
- dannyw 1mo agoModels are trained in highly distributed and parallel ways these days. It is entirely possible to run multiple concurrent post-training runs. When a frontier lap deploys a 1M RL gym rollout, these 1M environments are absolutely not talking to each other or interconnected. They individually generate traces and movements that can be then combined for post training.