5 ms·

I read it as it is impressive because smaller models 2.5T are squeezing similar returns as 10T models despite being 1/4th size not that there beyond 2T today th

by manquer 29d ago

I read it as it is impressive because smaller models 2.5T are squeezing similar returns as 10T models despite being 1/4th size not that there beyond 2T today the number or parameters do not have much meaning

verdverm 29d ago
or the latest qwen3.8 27B doing so well at ~1/100 the size of K3
- rnewme 29d ago
  What about general knowledge you can get out of it before hallucinations start?
  - walrus01 29d ago
    It did OK on schlongbench v1.0 (test of a specific niche word that doesn't make it into smaller LLMs) but it sure does love to count words https://pastes.io/r8F1AY8h https://pastes.io/r8F1AY8h
  - verdverm 29d ago
    I do not rely on any LLM of any size for general knowledge baked into the weights, they all hallucinate and that is the wrong way to hold them imo I think there is some merit in that smaller models cannot memorize so much of the training data, i.e. that they are less likely to do copyright infringement, and by analogy not having memorized SDK / API surfaces that have since changed from the training data
    - walrus01 29d ago
      > I do not rely on any LLM of any size for general knowledge baked into the weights You have to rely on it to a certain level for agentic/coding work, presuming that's the general subject we're talking about here... For instance I recently encountered a project where it would have been a lot worse if the LLM didn't already know "what is" xterm.js and a bunch of its associated npm-related/node related software. If it was still smart but had to google and find results for everything it would have been a lot more time consuming and risked sending it down a wrong path.
      - verdverm 28d ago
        for sure, there is a minimum size and knowledge base that is required to be useful at the same time, search may find newer or better alternatives, and you can always specify specific technologies you want to use, I typically do this when starting a new project
  - stymaar 29d ago
    Storing general knowledge in VRAM has always been a dumb idea in the first place.
  - Balinares 28d ago
    Qwen 3.8 27B beats Opus, Fable and GPT 5.6 by a comfortable margin on the AA-Omniscience Hallucination Rate benchmark.

by manquer 29d ago

verdverm 29d ago

or the latest qwen3.8 27B doing so well at ~1/100 the size of K3

rnewme 29d ago
What about general knowledge you can get out of it before hallucinations start?
- walrus01 29d ago
  It did OK on schlongbench v1.0 (test of a specific niche word that doesn't make it into smaller LLMs) but it sure does love to count words https://pastes.io/r8F1AY8h https://pastes.io/r8F1AY8h
- verdverm 29d ago
  I do not rely on any LLM of any size for general knowledge baked into the weights, they all hallucinate and that is the wrong way to hold them imo I think there is some merit in that smaller models cannot memorize so much of the training data, i.e. that they are less likely to do copyright infringement, and by analogy not having memorized SDK / API surfaces that have since changed from the training data
  - walrus01 29d ago
    > I do not rely on any LLM of any size for general knowledge baked into the weights You have to rely on it to a certain level for agentic/coding work, presuming that's the general subject we're talking about here... For instance I recently encountered a project where it would have been a lot worse if the LLM didn't already know "what is" xterm.js and a bunch of its associated npm-related/node related software. If it was still smart but had to google and find results for everything it would have been a lot more time consuming and risked sending it down a wrong path.
    - verdverm 28d ago
      for sure, there is a minimum size and knowledge base that is required to be useful at the same time, search may find newer or better alternatives, and you can always specify specific technologies you want to use, I typically do this when starting a new project
- stymaar 29d ago
  Storing general knowledge in VRAM has always been a dumb idea in the first place.
- Balinares 28d ago
  Qwen 3.8 27B beats Opus, Fable and GPT 5.6 by a comfortable margin on the AA-Omniscience Hallucination Rate benchmark.