5 ms·
Is it actually that hard to make good models or is it just about the amount of resources you have to do training? (This is an actual question, I really don't kn
by lifeformed 3mo ago
Is it actually that hard to make good models or is it just about the amount of resources you have to do training? (This is an actual question, I really don't know.) I'm sure it's not trivial but does it really take world class secret knowledge to build off of the known existing techniques? I feel like there's tons of low hanging fruit still to explore, and time and resources are the limiting factor.
- MostlyStable 3mo agoThe gap between grok and Gemini to Claude and chatgpt suggests that yes it is that hard.
- arw0n 3mo agoI suspect that Grok has been ironically lobotomized by pressures to correct its political views. Similarly, I could imagine the Gemini folks working in a significantly more complex corporate climate, with different parts of Google pushing for different capability focuses. They are only lagging behind less than a year, so it isn't too large of a gap yet. That said, the fact that Anthropic is currently the top dog suggests that talent and execution is incredibly important. A year ago none of my normie friends new them, and when i suggested using Claude looked at me like when I recommend Linux.
- janalsncm 3mo agoThat shouldn’t affect Grok’ coding ability. How often are people discussing politics with Claude code? Writing decent code is just hard and it’s not just Grok.
- bwhiting2356 3mo agoIt affects their ability to hire and retain talent.
- janalsncm 3mo agoIf training a good model requires talent then that’s the answer to the question this thread is trying to answer: is training a good model actually that hard?
- buthowjejddjeu 3mo agoTalent to do.. what? This could mean a lot of things. Navigating astronomically huge fundamentally not so hard but still really tangly and hairy projects requiring both excellent short- and long-term vision in an overheated domain with angry people and lots of money is a skill all of its own.
- janalsncm 3mo agoTalent to train high quality LLMs, especially coding LLMs.
- black_knight 3mo agoWhy would these be independent?
- janalsncm 3mo agoMore specifically, political lobotomy shouldn’t affect coding ability.
- Discordian93 3mo agoYet empirically it does
- girvo 3mo agoYou’d be quite surprised, I think. Fine tuning a model on one axis can have drastic impacts on another that as a human we would expect to be completely unrelated.
- janalsncm 3mo agoI have never seen anyone argue that this cannot be overcome with more high quality RLVR data. The practical reality is that the Chinese and American models might have very different politics. But the most relevant factor in model performance is the quality and volume of training data, not ideology of the base model. Unless you are suggesting something very particular about the way Grok was neutered.
- Hamuko 3mo agoIt's all a bunch of weights isn't it? Why wouldn't fiddling with some parts of the weights have cascading effects?
- thot_experiment 3mo agoNot true, aggressive post training makes models notably dumber.
- KaiserPro 3mo ago> That shouldn’t affect Grok’ coding ability. If you are spending all your time having to re-train because the boss doesn;t like the output, it will hamper coding
- Oreb 3mo ago> A year ago none of my normie friends new them, and when i suggested using Claude looked at me like when I recommend Linux. Isn’t that still the case? Normies haven’t even heard about Claude, in my experience.
- buthowjejddjeu 3mo agoIn my experience it has improved a bit, but 90% of my normies still have no idea. (It was 100% before)
- khurs 3mo agoAll of the 11 grok co-founders alongside Elon quit: https://techcrunch.com/2026/03/28/elon-musks-last-co-founder-reportedly-leaves-xai/ https://techcrunch.com/2026/03/28/elon-musks-last-co-founder... so that will have hampered grok. Like Zuckerberg, top talent may not work with a polarising character if they disagree with his behaviour. Space focused talent don't have many choices aside from SpaceX but ai companies are a plenty and a top AI person can pick and choose.
- MostlyStable 3mo agoThe fact that you need top talent also suggests that it is indeed that hard
- IshKebab 3mo agoI dunno, if most of the top of a company quits it's extremely disruptive even if everyone else in the company is competent.
- deleted 3mo ago[deleted]
- fwipsy 3mo agoNot hard to be a fast follower. Lots of companies are ~6-9 months behind. Reaching the actual bleeding edge is much harder.
- khurs 3mo ago>Is it actually that hard to make good models Didn't take DeepSeek long. Or XAI to launch grok. If they have a top team and the money then appears to be a matter of a year or two? And one startup mentioned is Japanese not Chinese so they won't be banned from buying US tech.