6 ms·

I always thought it could be because volume-wise, most English prose is probably marketing copy and actual clickbait; so when you train on the entire Internet,

I always thought it could be because volume-wise, most English prose is probably marketing copy and actual clickbait; so when you train on the entire Internet, you get a troll adept at writing ads. Then people ask AdBot2000 to write a novel and are upset it reads like the next iPhone launch site.

astrange 15d ago
No, there's no reason chatbot behavior would have anything to do with frequency of text in pretraining.
Anon1096 15d ago
Nah, I think this is a common misunderstanding of how LLMs work, where people think that they mimic the pre-training data. Stylistically everything you see is an artifact of post-training, which is from reinforcement learning not from absorbing mass amounts of text. At some point a person or more recently a bot gave a thumbs up to an A/B tested response including em-dashes and claudisms galore.
- BoredomIsFun 15d ago
  > Stylistically everything you see is an artifact of post-training, It is still not exactly clear if it is true or not. Unless we have base "pt" snaphot of Claude we can't say one way or another. I've played a bit with base models of Nemo, Gemma etc and they all had tics, not much different from RLHFed instruct versions.
- ModernMech 15d ago
  So question then, why is it so hard to make an ai that doesn’t do these things? And why do Claude and ChatGPT have the same -isms? They’re both doing the same a/b post training with the same decisions?
  - idiotsecant 15d ago
    You don't blame the puddle for taking the shape of the hole.
  - cyclopeanutopia 15d ago
    It would require changing humans first.
- kridsdale1 15d ago
  Yes. This completely explains sycophancy at least.
- avereveard 15d ago
  There's layers, some of token selection is fingerprinting https://github.com/google-deepmind/synthid-text https://github.com/google-deepmind/synthid-text
  - ekidd 15d ago
    Yeah, but I understand that fingerprinting is essentially a pseudorandom overlay onto a pseudorandom base signal. And unless you have access to both the random number generators and the weights, I don't think you can detect it? So "fingerprinting" operates on a totally different and basically invisible level, as opposed to the obvious stylistic patterns that the average programmer can identify in about 2 sentences.
    - avereveard 14d ago
      Eh we can detect opumism and gptisms our brain are very good at pattern recognition even if subconscious

astrange 15d ago

No, there's no reason chatbot behavior would have anything to do with frequency of text in pretraining.

Anon1096 15d ago

Nah, I think this is a common misunderstanding of how LLMs work, where people think that they mimic the pre-training data. Stylistically everything you see is an artifact of post-training, which is from reinforcement learning not from absorbing mass amounts of text. At some point a person or more recently a bot gave a thumbs up to an A/B tested response including em-dashes and claudisms galore.

BoredomIsFun 15d ago
> Stylistically everything you see is an artifact of post-training, It is still not exactly clear if it is true or not. Unless we have base "pt" snaphot of Claude we can't say one way or another. I've played a bit with base models of Nemo, Gemma etc and they all had tics, not much different from RLHFed instruct versions.
ModernMech 15d ago
So question then, why is it so hard to make an ai that doesn’t do these things? And why do Claude and ChatGPT have the same -isms? They’re both doing the same a/b post training with the same decisions?
- idiotsecant 15d ago
  You don't blame the puddle for taking the shape of the hole.
- cyclopeanutopia 15d ago
  It would require changing humans first.
kridsdale1 15d ago
Yes. This completely explains sycophancy at least.
avereveard 15d ago
There's layers, some of token selection is fingerprinting https://github.com/google-deepmind/synthid-text https://github.com/google-deepmind/synthid-text
- ekidd 15d ago
  Yeah, but I understand that fingerprinting is essentially a pseudorandom overlay onto a pseudorandom base signal. And unless you have access to both the random number generators and the weights, I don't think you can detect it? So "fingerprinting" operates on a totally different and basically invisible level, as opposed to the obvious stylistic patterns that the average programmer can identify in about 2 sentences.
  - avereveard 14d ago
    Eh we can detect opumism and gptisms our brain are very good at pattern recognition even if subconscious