33 ms·
If you check failure section of their paper, they also tried other methods like MCTS and PRM which is what other labs have been obsessing about but couldn't mov
by attentionmech 2y ago
If you check failure section of their paper, they also tried other methods like MCTS and PRM which is what other labs have been obsessing about but couldn't move on from (that includes bigshots). Only team which I am aware which tried verifiable rewards is tulu but they didn't scaled it up and just left it there.
This sort of thing imo is similar to what openAI did with transformer architecture i.e. google invented it but couldn't scale it in the right direction and deepmind got busy with atari games. They had all the pieces still openai could do it. It seems to be it comes down to research leadership in what methods to choose to invest in. But yeah, the budgets big labs have, they can easily try 10 different techniques and brute force it all but seems like they are too opinionated in methods and less urgent on outcomes.
[paper] https://arxiv.org/pdf/2501.12948 https://arxiv.org/pdf/2501.12948
[tulu] https://x.com/hamishivi/status/1881394117810500004 https://x.com/hamishivi/status/1881394117810500004
- attentionmech 2y agoI found the following thread more insightful than my original comment (wish I could edit that one). A research explains why RL didn't work before this: https://x.com/its_dibya/status/1883595705736163727 https://x.com/its_dibya/status/1883595705736163727
- qnleigh 2y agoThat's interesting. I suppose it could even be possible to test his theories. Just applied the exact same training methodology to smaller models or slightly easier problems and study what happens.
- attentionmech 2y agopeople already did: https://x.com/karpathy/status/1884678601704169965 https://x.com/karpathy/status/1884678601704169965
- krackers 2y agoRelated: https://twitter.com/voooooogel/status/1884089601901683088#m https://twitter.com/voooooogel/status/1884089601901683088#m Also https://epoch.ai/gradient-updates/how-has-deepseek-improved-the-transformer-architecture https://epoch.ai/gradient-updates/how-has-deepseek-improved-... has a summary of all the architectural improvements DeepSeek made to increase performance.