23 ms·

> Why not just say "incoherent output"? Because the biggest problem with hallucinations is that the output is usually coherent but factually incorrect. I agree

> Why not just say "incoherent output"? Because the biggest problem with hallucinations is that the output is usually coherent but factually incorrect. I agree that "hallucination" isn't the best word for it... perhaps something like "confabulation" is better.

linguistbreaker 2y ago
I appreciated a post on here recently that likened AI hallucination to 'bullshitting'. It's coherent, even plausible output without any regard for the truth.
- wkat4242 2y ago
  While I have absolutely no issues with the word "shit" in popular terms, I'd normally like to reserve it for situations where there's actually intended malice like in "enshittification". Rather than just an imperfect technology as we have here. Many people object to the term enshittification for foul-mouthing reasons but I think it covers it very well because the principle it covers is itself so very nasty. But that's not at all the case here.
  - pessimizer 2y ago
    "Bullshitting" isn't a new piece of jargon, it's a common English word of many decades vintage, and is being used in its dictionary sense here.
- MattPalmer1086 2y ago
  More true to say that all output is bullshitting, not just the ones we call hallucinations. Some of it is true, some isn't. The model doesn't know or care.
nl 2y ago
And we use "hallucination" because in the ancient times when generative AI meant image generation models would "hallucinate" extra fingers etc. The behavior of text models is similar enough that the wording stuck, and it's not all that bad.
- darby_nine 2y ago
  That was a misnomer—hallucination refers to perception, not generation. Completely misled an entire generation of people.
- Raphaellll 2y ago
  "hallucination" was coined in the context of text generating RNNs. Specifically in this blog post by Karpathy in 2015: https://karpathy.github.io/2015/05/21/rnn-effectiveness/ https://karpathy.github.io/2015/05/21/rnn-effectiveness/

linguistbreaker 2y ago

I appreciated a post on here recently that likened AI hallucination to 'bullshitting'. It's coherent, even plausible output without any regard for the truth.

wkat4242 2y ago
While I have absolutely no issues with the word "shit" in popular terms, I'd normally like to reserve it for situations where there's actually intended malice like in "enshittification". Rather than just an imperfect technology as we have here. Many people object to the term enshittification for foul-mouthing reasons but I think it covers it very well because the principle it covers is itself so very nasty. But that's not at all the case here.
- pessimizer 2y ago
  "Bullshitting" isn't a new piece of jargon, it's a common English word of many decades vintage, and is being used in its dictionary sense here.
MattPalmer1086 2y ago
More true to say that all output is bullshitting, not just the ones we call hallucinations. Some of it is true, some isn't. The model doesn't know or care.

nl 2y ago

And we use "hallucination" because in the ancient times when generative AI meant image generation models would "hallucinate" extra fingers etc. The behavior of text models is similar enough that the wording stuck, and it's not all that bad.

darby_nine 2y ago
That was a misnomer—hallucination refers to perception, not generation. Completely misled an entire generation of people.
Raphaellll 2y ago
"hallucination" was coined in the context of text generating RNNs. Specifically in this blog post by Karpathy in 2015: https://karpathy.github.io/2015/05/21/rnn-effectiveness/ https://karpathy.github.io/2015/05/21/rnn-effectiveness/