10 ms·

There's definitely a few echo chambers around AI, but it's definitely not something that "just techies" are onto. ChatGPT made some waves at the end of last ye

There's definitely a few echo chambers around AI, but it's definitely not something that "just techies" are onto. ChatGPT made some waves at the end of last year. My in-laws were wanting to talk to (at) me about it at Christmas. There's plenty of awareness outside of the tech circles, but most of the discussion (both out and in of the tech world) seems to miss what LLMs actually _are_. The reason why ChatGPT was impressive to me wasn't the "realism" of the responses... It was how quickly it could classify and chain inputs/outputs. It's super impressive tech, but like... It's not AI. As accurate as it may ever seem, it's simply not actually aware of what it's saying. "Hallucinations" is a fun term, but it's not hallucinating information, it's just guessing at the next token to write because that's all it ever does. If it was "intelligent" it would be able to recognise a limitation in its knowledge and _not_ hallucinate information. But it can't. Because it doesn't know anything. Correct answers are just as hallucinatory as incorrect answers because it's the exact same mechanism that produces them - there's just better probabilities.

adrianN 3y ago
AlphaGo is not aware that it’s playing a game either, but it’s better than humans at it. Awareness is not necessary to make people lose their jobs.
- sam_lowry_ 3y ago
  > Awareness is not necessary Wasn't it the plot of a sci-fi novel by Vernor Vinge or someone at least as popular?
  - fiveg 3y ago
    You might be thinking of Blindsight by Peter Watts. Great book.
- samuellevy 3y ago
  I don't really know anything about AlphaGo. There's more types of "AI" than LLMs, but that's not really the point. You don't need AI for people to lose their jobs... but nobody is losing their jobs to AlphaGo, and in the grand scheme of things it's unlikely that people are going to lose their jobs to GPT, too.
  - adrianN 3y ago
    If you make people who produce text 25% more productive you can fire one in four and increase your profits.
InvertedRhodium 3y ago
In your opinion, how does the "hallucination" issue differ from the same behaviour we see in humans? I don't claim or believe that any LLM is actually intelligent. It just seems that we (at least on an individual basis) can also meet the criteria outlined above. I know plenty of people who are confidently incorrect and appear unwilling to learn or accept their own limitations, myself included. In my opinion, even if we did have AGI it would still exhibit a lot of our foibles given that we'd be the only ones teaching it.
- dmarchand90 3y ago
  Yeah I've never gotten this argument at all. "Humans aren't actually intelligent they're just machines designed to optimize their probability of reproducing "
- f4f4f4f43f 3y ago
  Yes, you may be. But you still have an internal world model - through conditioning or otherwise that you're playing off against. An LLM doesn't have that. It's very impressive parlour trick (and of course a lot more), but it's use is hence limited (albeit massive) to that. Chaining and context assists resolving that to some extent, but it's a limited extent. That's the argument anyway, that doesn't mean it's not incredibly impressive, but comparing it to human self-awareness, however small, isn't a fair comparison. It's next token prediction, which is why it does classification so well.
- samuellevy 3y ago
  So humans have a level of knowledge, understanding, and reasoning ability that LLMs simply don't have. I'm writing a response to you right now, and I "know" a certain amount of information about the world. That knowledge has limits, and I can expand it, I can forget it, all sorts of things... "Hallucination" is a term that works well for actual intelligence - when you "know" something that isn't true, and has no path of reasoning, you might have hallucinated the base "knowledge". But that doesn't really work for LLMs, because there's no knowledge at all. All they're doing is picking the next most likely token based on the probabilities. If you interrogate something that the training data covers thoroughly, you'll get something that is "correct", and that's to be expected because there's a lot of probabilities pointing to the "next token" being the right one... but as you get to the edge of the training data, the "next token" is less likely to be correct. As a thought experiment, imagine that you're given a book with every possible or likely sequence of coloured circles, triangles, and squares. None of them have meaning to you, they're just colours and shapes that are in random seeming sequences, but there's a frequency to them. "Red circle, blue square, gren triangle" is a much more common sequence than "red circle, blue square, black triangle", so if someone hands you a piece of paper with "red circle, blue square", you can reasonably guess that what they want back is a green triangle. Expand the model a bit more, and you notice that "rc bs gt" is pretty common, but if there's a yellow square a few symbols before with anything in between, then the triangle is usually black. Thus the response to the sequence "red circle, blue square" is usually "green triangle", but "black circle, yellow square, grey circle, red circle, blue square" is modified by the yellow square, and the response is "black triangle"... but you still don't know what any of these things _mean_. When you get to a sequence that isn't covered directly by the training data, you just follow the process with the information that you _do_ have. You get "red triangle, blue square" and while you've not encountered that sequence before, "green" _usually_ comes after "red, blue", and "circle" is _usually_ grouped with "triangle, square", so a reasonable response is "green circle"... but we don't know, we're just guessing based on what we've seen. That's the thing... the process is exactly the same whether the sequence has been seen before or not. You're not _hallucinating_ the green circle, you're just picking based on probabilities. LLMs are doing effectively this, but at massive scale with an unthinkably large dataset as training data. Because there's so much data of _humans talking to other humans_, ChatGPT has a lot of probabilities that make human-sounding responses... It's not an easy concept to get across, but there's a fundamental difference between "knowing a thing and being able to discuss it" and "picking the next token based on the probabilities gleaned from inspecting terabytes of text, without understanding what any single token means"
Obscurity4340 3y ago
Not to be difficult but wouldn't "confabulating" be a preferable description for this behaviour? Hallucinating doesn't quite feel right but I can't exactly articulate why confabulate is superior in this context
- samuellevy 3y ago
  Nah, my issue with both terms is that they imply that when the answer is "correct" that's because the LLM "knows" the correct answer, and when it's wrong it's just a brain fart. It doesn't matter if the output is correct or not, the process for producing it is identical, and the model has the exact same amount of knowledge about what it's saying... which is to say "none". This isn't a case of "it's intelligent, but it gets muddled up sometimes". It's more of the case that it's _always_ muddled up, but it's accidentally correct a lot of the time.
  - bondarchuk 3y ago
    >It doesn't matter if the output is correct or not, the process for producing it is identical I don't see how this differs from a human earnestly holding a mistaken belief.
- bondarchuk 3y ago
  "Hallucinating" (normally) means having a subjective experience of the same type as a sensory perception, without the presence of a stimulus that would normally cause such a perception. I agree it's weird to apply this term to an LLM because it doesn't really have sensory perception at all. Of course it has text input, but if you consider that to be equivalent to sensory perception (which I'd be open to) then a hallucination would mean to act as if something is in the text input when it really isn't, which is not how people use the term. You could also consider all the input it got during training as its sensory perception (also arguable IMHO), but then a proper hallucination would entail some mistaken classification of the input resulting in incorrect training, which is also not really what's going on I think. Confabulation is a much more accurate term indeed, going by the first paragraph of wikipedia.
narag 3y ago
It's not AI. As accurate as it may ever seem, it's simply not actually aware of what it's saying. Conflating intelligence and awareness seems to me the biggest confusion around this topic. When non-technical people ask me about it, I ask them to consider three questions: - is alive? - thinks? - can speak (and understand)? A plant, microbe, primitive animals... are alive, don't think, can't speak. A dog, a monkey... are alive, think, can't speak. A human is alive, thinks, can speak. These things aren't alive, think, can speak. I know some of the above will be controversial, but clicks for most people, that agree: if you have a dog, you know what I mean whith "a dog thinks". Not with words, but they're capable intricate reasoning and strategies. Intelligence can be mechanical, the same as force. For a man from the ancient times, the concept of an engine would have been weird. Only live beings were thought to move on their own. When a physical process manifested complex behaviour, they said that a spirit was behind it. Intelligence doesn't need awareness. You can have disembodied pieces of intelligence. That's what Google, Facebook, etc. have been doing for a long time. They're AI companies. It doesn't help with the confusion that speaking is a harder condition than thinking and thinking seems to be harder than being alive: "these things aren't alive so they can't think" but they speak, so...
- samuellevy 3y ago
  Ehh... my dog is alive, thinks, and "speaks" in a manner - not a cute term for barking, but he communicates (with relatively high effectiveness) his wants and desires. Maybe not using human words, but he certainly has his own sort of crude language, as does my cat. The problem is that LLMs aren't alive, and they _don't think_. The speaking is arguable.
- veidr 3y ago
  You might be onto something (or not, I'm not sure), but its extremely well-documented that both dogs and monkeys can speak. They can't speak English like a human, but they both can understand a good deal of English, and they both can speak in their own ways (and understand the speaking of others). I think the key thing about these LLMs is that they upend the notion that speaking requires thinking/understanding/intelligence. They can "speak", if you mean emit coherent sentences and paragraphs, really well. But there is no understanding of anything, nor thinking, nor what most people would understand as intelligence behind that speaking. I think that is probably new. I can't think of anything that could speak on this level, and yet be completely and obviously (if you give it like, an hour of back and forth conversation) devoid of intelligence or thinking. I think that's what makes people have fantastical notions about how intelligent or useful LLMs are. We're conditioned by the entirety of human history to equate such high-quality "speech" with intelligence. Now we've developed a slime mold that can write novels. But I think human society will adapt quickly, and recalibrate that association.

adrianN 3y ago

AlphaGo is not aware that it’s playing a game either, but it’s better than humans at it. Awareness is not necessary to make people lose their jobs.

sam_lowry_ 3y ago
> Awareness is not necessary Wasn't it the plot of a sci-fi novel by Vernor Vinge or someone at least as popular?
- fiveg 3y ago
  You might be thinking of Blindsight by Peter Watts. Great book.
samuellevy 3y ago
I don't really know anything about AlphaGo. There's more types of "AI" than LLMs, but that's not really the point. You don't need AI for people to lose their jobs... but nobody is losing their jobs to AlphaGo, and in the grand scheme of things it's unlikely that people are going to lose their jobs to GPT, too.
- adrianN 3y ago
  If you make people who produce text 25% more productive you can fire one in four and increase your profits.

InvertedRhodium 3y ago

In your opinion, how does the "hallucination" issue differ from the same behaviour we see in humans? I don't claim or believe that any LLM is actually intelligent. It just seems that we (at least on an individual basis) can also meet the criteria outlined above. I know plenty of people who are confidently incorrect and appear unwilling to learn or accept their own limitations, myself included. In my opinion, even if we did have AGI it would still exhibit a lot of our foibles given that we'd be the only ones teaching it.

dmarchand90 3y ago
Yeah I've never gotten this argument at all. "Humans aren't actually intelligent they're just machines designed to optimize their probability of reproducing "
f4f4f4f43f 3y ago
Yes, you may be. But you still have an internal world model - through conditioning or otherwise that you're playing off against. An LLM doesn't have that. It's very impressive parlour trick (and of course a lot more), but it's use is hence limited (albeit massive) to that. Chaining and context assists resolving that to some extent, but it's a limited extent. That's the argument anyway, that doesn't mean it's not incredibly impressive, but comparing it to human self-awareness, however small, isn't a fair comparison. It's next token prediction, which is why it does classification so well.
samuellevy 3y ago
So humans have a level of knowledge, understanding, and reasoning ability that LLMs simply don't have. I'm writing a response to you right now, and I "know" a certain amount of information about the world. That knowledge has limits, and I can expand it, I can forget it, all sorts of things... "Hallucination" is a term that works well for actual intelligence - when you "know" something that isn't true, and has no path of reasoning, you might have hallucinated the base "knowledge". But that doesn't really work for LLMs, because there's no knowledge at all. All they're doing is picking the next most likely token based on the probabilities. If you interrogate something that the training data covers thoroughly, you'll get something that is "correct", and that's to be expected because there's a lot of probabilities pointing to the "next token" being the right one... but as you get to the edge of the training data, the "next token" is less likely to be correct. As a thought experiment, imagine that you're given a book with every possible or likely sequence of coloured circles, triangles, and squares. None of them have meaning to you, they're just colours and shapes that are in random seeming sequences, but there's a frequency to them. "Red circle, blue square, gren triangle" is a much more common sequence than "red circle, blue square, black triangle", so if someone hands you a piece of paper with "red circle, blue square", you can reasonably guess that what they want back is a green triangle. Expand the model a bit more, and you notice that "rc bs gt" is pretty common, but if there's a yellow square a few symbols before with anything in between, then the triangle is usually black. Thus the response to the sequence "red circle, blue square" is usually "green triangle", but "black circle, yellow square, grey circle, red circle, blue square" is modified by the yellow square, and the response is "black triangle"... but you still don't know what any of these things _mean_. When you get to a sequence that isn't covered directly by the training data, you just follow the process with the information that you _do_ have. You get "red triangle, blue square" and while you've not encountered that sequence before, "green" _usually_ comes after "red, blue", and "circle" is _usually_ grouped with "triangle, square", so a reasonable response is "green circle"... but we don't know, we're just guessing based on what we've seen. That's the thing... the process is exactly the same whether the sequence has been seen before or not. You're not _hallucinating_ the green circle, you're just picking based on probabilities. LLMs are doing effectively this, but at massive scale with an unthinkably large dataset as training data. Because there's so much data of _humans talking to other humans_, ChatGPT has a lot of probabilities that make human-sounding responses... It's not an easy concept to get across, but there's a fundamental difference between "knowing a thing and being able to discuss it" and "picking the next token based on the probabilities gleaned from inspecting terabytes of text, without understanding what any single token means"

Obscurity4340 3y ago

Not to be difficult but wouldn't "confabulating" be a preferable description for this behaviour? Hallucinating doesn't quite feel right but I can't exactly articulate why confabulate is superior in this context

samuellevy 3y ago
Nah, my issue with both terms is that they imply that when the answer is "correct" that's because the LLM "knows" the correct answer, and when it's wrong it's just a brain fart. It doesn't matter if the output is correct or not, the process for producing it is identical, and the model has the exact same amount of knowledge about what it's saying... which is to say "none". This isn't a case of "it's intelligent, but it gets muddled up sometimes". It's more of the case that it's _always_ muddled up, but it's accidentally correct a lot of the time.
- bondarchuk 3y ago
  >It doesn't matter if the output is correct or not, the process for producing it is identical I don't see how this differs from a human earnestly holding a mistaken belief.
bondarchuk 3y ago
"Hallucinating" (normally) means having a subjective experience of the same type as a sensory perception, without the presence of a stimulus that would normally cause such a perception. I agree it's weird to apply this term to an LLM because it doesn't really have sensory perception at all. Of course it has text input, but if you consider that to be equivalent to sensory perception (which I'd be open to) then a hallucination would mean to act as if something is in the text input when it really isn't, which is not how people use the term. You could also consider all the input it got during training as its sensory perception (also arguable IMHO), but then a proper hallucination would entail some mistaken classification of the input resulting in incorrect training, which is also not really what's going on I think. Confabulation is a much more accurate term indeed, going by the first paragraph of wikipedia.

narag 3y ago

It's not AI. As accurate as it may ever seem, it's simply not actually aware of what it's saying. Conflating intelligence and awareness seems to me the biggest confusion around this topic. When non-technical people ask me about it, I ask them to consider three questions: - is alive? - thinks? - can speak (and understand)? A plant, microbe, primitive animals... are alive, don't think, can't speak. A dog, a monkey... are alive, think, can't speak. A human is alive, thinks, can speak. These things aren't alive, think, can speak. I know some of the above will be controversial, but clicks for most people, that agree: if you have a dog, you know what I mean whith "a dog thinks". Not with words, but they're capable intricate reasoning and strategies. Intelligence can be mechanical, the same as force. For a man from the ancient times, the concept of an engine would have been weird. Only live beings were thought to move on their own. When a physical process manifested complex behaviour, they said that a spirit was behind it. Intelligence doesn't need awareness. You can have disembodied pieces of intelligence. That's what Google, Facebook, etc. have been doing for a long time. They're AI companies. It doesn't help with the confusion that speaking is a harder condition than thinking and thinking seems to be harder than being alive: "these things aren't alive so they can't think" but they speak, so...

samuellevy 3y ago
Ehh... my dog is alive, thinks, and "speaks" in a manner - not a cute term for barking, but he communicates (with relatively high effectiveness) his wants and desires. Maybe not using human words, but he certainly has his own sort of crude language, as does my cat. The problem is that LLMs aren't alive, and they _don't think_. The speaking is arguable.
veidr 3y ago
You might be onto something (or not, I'm not sure), but its extremely well-documented that both dogs and monkeys can speak. They can't speak English like a human, but they both can understand a good deal of English, and they both can speak in their own ways (and understand the speaking of others). I think the key thing about these LLMs is that they upend the notion that speaking requires thinking/understanding/intelligence. They can "speak", if you mean emit coherent sentences and paragraphs, really well. But there is no understanding of anything, nor thinking, nor what most people would understand as intelligence behind that speaking. I think that is probably new. I can't think of anything that could speak on this level, and yet be completely and obviously (if you give it like, an hour of back and forth conversation) devoid of intelligence or thinking. I think that's what makes people have fantastical notions about how intelligent or useful LLMs are. We're conditioned by the entirety of human history to equate such high-quality "speech" with intelligence. Now we've developed a slime mold that can write novels. But I think human society will adapt quickly, and recalibrate that association.