6 ms·
There are many indications that model progress is slowing down, so that is not entirely accurate.
by civvv 4mo ago
There are many indications that model progress is slowing down, so that is not entirely accurate.
- StrauXX 4mo agoWhich indications are that?
- overfeed 4mo agoInvestment dollars.
- dzhiurgis 4mo agoSource for that claim?
- deleted 4mo ago[deleted]
- lionkor 4mo agoNobody is releasing NEW models
- taneq 4mo agoThe standard networking connection has been called “Ethernet” for more than thirty years, so networking has stagnated, right?
- SlinkyOnStairs 4mo agoIf higher bandwidth networking consisted primarily running more and more ethernet lines in parallel, you would most certainly agree that "networking has stagnated". "Reasoning" and now "Agentic" AI systems are not some fundamental improvement on LLMs, they're just running roughly the same prior-gen LLMS, multiple times. Hence the conclusion that LLM improvement has slowed down, if not stagnated entirely, and that we should not expect the improvements of switching to these "reasoning" systems to keep happening.
- p1esk 4mo agoFrom TFA: “ChatGPT came up with an idea which is original and clever. It is the sort of idea I would be very proud to come up with after a week or two of pondering, and it took ChatGPT less than an hour to find and prove”
- SlinkyOnStairs 4mo agoYou misunderstand. I'm not saying that Reasoning/Agentic systems aren't better. I'm saying they're not an advancement in the tech in the way GPT 1 through 3 were. They're a different kind of improvement. And as such the rate improvement cannot just be extrapolated into the future.
- p1esk 4mo agoGPT1 through GPT3 advancement were exactly like using more Ethernet cables in parallel. All interesting conceptual breakthroughs came after GPT3: RL and reasoning being the main ones.
- kstenerud 4mo agoWhat constitutes a NEW model for the purposes of calculating progress?
- aspenmartin 4mo ago…not only is this not true but it also doesn’t matter. Why would this indicate performance saturating?
- GardenLetter27 4mo agoWhat? DeepSeekV3 just came out and is incredible for the price. Mythos is also half-released.
- nozzlegear 4mo agoUntil you or I can actually use Mythos in Claude without an nda or other strings attached, Mythos is not released and is just an effective marketing tool for Anthropic.
- pixl97 4mo agoAt least to me this is a pretty sour grapes take. There are all kinds of released products that are expensive or need an NDA. You're just too poor to afford it. But make no mistakes there are governments using this in mass and likely against you.
- nxobject 4mo agoI think that’s worthy of at least sour grapes, too.
- nicoburns 4mo agoThe cost factors on the new models compared to the old models.
- bdelmas 4mo agoYou are mixing cost and progress. It’s not because it’s more and more expensive that progress is slowing down by itself.
- nicoburns 4mo agoThey are intrinsically linked beyond a certain point. If we're making progress but costs are spiraling exponentially then it stands to reason that we will soon reach a point where we can no longer afford the increasing costs and thus progress will slow. (barring some breakthrough that reduces costs, which of course may happen, but for which recent model improvements are not strong evidence of)
- jeremyjh 4mo agoQwen3.6 9B is as good as GPT-4o and runs on my M2 MacBook Air. Models are getting stronger and less costly at the same time, but these are somewhat separate branches of research. Frontier labs are spending more because they are still getting marginal returns and there is more capacity to spend than there was a year ago.
- gertop 4mo agoQwen 3.6 9B doesn't exist. If you meant 3.5 9B and you truly believe it's as good as 4o then I can only assume you have a very basic use case.
- jeremyjh 4mo agoYou are right, I was mistaken about the version. I evaluated it in general chat assistant prompts plucked from my history across a range of topics but did not use it for coding - there was never a time when I thought 4o was “good enough” for agentic coding.
- aspenmartin 4mo agoPlease be specific because outside of anecdotal blog posts by people who don’t know what they’re talking about it’s not true. Look at scaling laws, composite benchmarks from the epoch capability index, nothing at all suggests “model progress is slowing down”
- CuriouslyC 4mo agoModel progress at spitting out unhallucinated facts is slowing down hard. Model progress at solving hard math challenges/programming tasks doesn't seem to be slowing down that I can tell.