6 ms·
> Since these companies can’t improve their AI models without fresh data created by human beings Totally wrong. Self-play dates back to Arthur Samuel in the 19
by jmmcd 5mo ago
> Since these companies can’t improve their AI models without fresh data created by human beings
Totally wrong. Self-play dates back to Arthur Samuel in the 1950s and RL with verifiable rewards is a key part of training the most advanced models today.
- cubefox 5mo agoCurrent models don't yet use RLVR with self-play though, at least as far as we know. They use RLVR with large numbers of manually created RL environments. But they will probably use self-play soon. See https://www.amplifypartners.com/blog-posts/self-play-and-autocurricula-in-the-age-of-agents https://www.amplifypartners.com/blog-posts/self-play-and-aut...
- rdedev 5mo agoNot totally wrong. Self play works well with if your problem can be easily simulated in an RL environment where the model can easily explore different states. RLHF or similar techniques is not that since we don't have exactly have a simulation environment for language modelling Right now there are companies which hire software devs or data scientists to just solve a bunch of random problems so that they can generate training data for an LLM model. Why would they be in business if self play can work out so well?
- vidarh 5mo ago> Why would they be in business if self play can work out so well? Because it is still cheaper.
- notpachet 5mo ago> Right now there are companies which hire software devs or data scientists to just solve a bunch of random problems so that they can generate training data for an LLM model. Sounds like Macrodata Refinement.