6 ms·
It's rumored fable is around that 10T number
by ewild 1mo ago
It's rumored fable is around that 10T number
- walrus01 1mo agoIf this is true, it's even more impressive that some of the open weight models that are <3.5T in size, approx 33% of its size, are within a few points of it in the artificial analysis leaderboard.
- habosa 1mo agoGLM 5.3 is "only" 753B parameters. Much much smaller.
- manquer 1mo agoNot necessarily, there could be diminishing returns on mere parameters count . There is nothing to say for example a 1 Quadrillion parameter model will be vastly more intelligent than current SOTA especially since new training data is largely synthetic today
- HDBaseT 1mo agoThat's precisely what he is saying, there is diminishing returns (or optimization left on the table).
- manquer 1mo agoI read it as it is impressive because smaller models 2.5T are squeezing similar returns as 10T models despite being 1/4th size not that there beyond 2T today the number or parameters do not have much meaning
- verdverm 1mo agoor the latest qwen3.8 27B doing so well at ~1/100 the size of K3
- rnewme 1mo agoWhat about general knowledge you can get out of it before hallucinations start?
- walrus01 1mo agoIt did OK on schlongbench v1.0 (test of a specific niche word that doesn't make it into smaller LLMs) but it sure does love to count words https://pastes.io/r8F1AY8h https://pastes.io/r8F1AY8h
- verdverm 1mo agoI do not rely on any LLM of any size for general knowledge baked into the weights, they all hallucinate and that is the wrong way to hold them imo I think there is some merit in that smaller models cannot memorize so much of the training data, i.e. that they are less likely to do copyright infringement, and by analogy not having memorized SDK / API surfaces that have since changed from the training data
- walrus01 29d ago> I do not rely on any LLM of any size for general knowledge baked into the weights You have to rely on it to a certain level for agentic/coding work, presuming that's the general subject we're talking about here... For instance I recently encountered a project where it would have been a lot worse if the LLM didn't already know "what is" xterm.js and a bunch of its associated npm-related/node related software. If it was still smart but had to google and find results for everything it would have been a lot more time consuming and risked sending it down a wrong path.
- verdverm 29d agofor sure, there is a minimum size and knowledge base that is required to be useful at the same time, search may find newer or better alternatives, and you can always specify specific technologies you want to use, I typically do this when starting a new project
- 29d ago
- mlmonkey 1mo agoYou want to take a look at the "Scaling Laws" paper, so you can extrapolate from these numbers.
- stymaar 29d agoThis paper, as well as the Chinchilla one, aged like milk though.
- scosman 29d agoAnd GLM is only 0.7T! But these labs distill off the larger models. Both officially at the labs with the big ones, and unofficially. We need the giant models to get the smaller models.
- johnnyApplePRNG 1mo agoFable is most definitely nowhere near 10T. The cost to train and infer that would be insane, even by today's standards.
- nl 1mo agoFable is strongly believed to be around 10T. The most conservative estimate I've seen is 8T. Eg: https://www.reuters.com/technology/bytedance-targets-mega-ai-model-nearing-anthropics-mythos-ft-reports-2026-08-07/ https://www.reuters.com/technology/bytedance-targets-mega-ai... That reports Mythos as 8T and Fable as 5T, but I think they mean Opus as 5T, which is widely known, eg: https://eu.36kr.com/en/p/3760679047267075?ref=explainx https://eu.36kr.com/en/p/3760679047267075?ref=explainx Both Grok and Bytedance are training 10T models.
- andai 1mo agoWasn't Opus ~1.5T and Fable is about twice that?
- stymaar 29d agoThe fact that Musk claims Opus is 5T to justify why Grok is far behind should be taken with a massive grain of salt given he's a recidivist mythomaniac. Honestly if Opus is 5T parameters while being matched by the biggest open models that are at least twice smaller, it would mean that the US is already behind China in the AI race, despite a significant edge in compute.
- WinstonSmith84 29d agoYes. And Opus goes a very long way compared to Fable, Anthropic isn't doing any favour, it's clearly just 2 models with a very different amount of parameters.
- nl 29d agoThe open models don't really match Opus. For example I regularly do Fable+Opus agentic coding runs over 24 hours without intervention. I think I've had GLM do a run that was a few hours. That's the closest I've had an open model come on that kind of work.