5 ms·

That's correct. LLMs are plausible sentence generators, they don't "understand"* their runtime environment (or any of their other input) and they're not qualifi

That's correct. LLMs are plausible sentence generators, they don't "understand"* their runtime environment (or any of their other input) and they're not qualified to answer your questions. The companies providing these LLMs to users will typically provide a qualification along these lines, because LLMs tend to make up ("hallucinate", in the industry vernacular) outputs that are plausibly similar to the input text, even if they are wildly and obviously wrong and complete nonsense to boot. Obviously, people find some value in some output of some LLMs. I've enjoyed the coding autocomplete stuff we have at work, it's helpful and fun. But "it's not qualified to answer my questions" is still true, even if it occasionally does something interesting or useful anyway. *- this is a complicated term with a lot of baggage, but fortunately for the length of this comment, I don't think that any sense of it applies here. An LLM doesn't understand its training set any more than the mnemonic "ETA ONIS"** understands the English language. **- a vaguely name-shaped presentation of the most common letters in the English language, in descending order. Useful if you need to remember those for some reason like guessing a substitution cypher.

ethbr1 3y ago
That's not entirely accurate. LLMs encode some level of understanding of their training set. Whether that's sufficient for a specific purpose, or sufficiently comprehensive to generate side effects, is an open question. * Caveat: with regards to introspection, this also assumes it's not specifically guarded against and opaquely lying.
ekianjo 3y ago
> plausible sentence generators, they don't "understand"* their runtime environment Exactly like humans dont understand how their brain works
- zlg_codes 3y ago
  We've put an awfully lot of effort into figuring that out, and have some answers. Much of the problems in exploring the brain are ethical because people tend to die or suffer greatly if we experiment on them. Unlike LLMs, which are built by humans and have literal source code and manuals and SOPs and shit. Their very "body" is a well-documented digital machine. An LLM trying to figure itself out has MUCH less trouble than a human figuring itself out.
CamperBob2 3y ago
If you can watch the video demo of this release, or for that matter the Attenborough video, and still claim that these things lack any form of "understanding," then your imagination is either a lot weaker than mine, or a lot stronger. Behavior indistinguishable from understanding is understanding. Sorry, but that's how it's going to turn out to work.
- zlg_codes 3y ago
  Have you considered that mankind simply trained itself on the wrong criteria on detecting understanding? Why are people so eager to believe that electric rocks can think?
  - CamperBob2 3y ago
    Why are people so eager to believe that people can? When it comes to the definitions of concepts like sentience, consciousness, thinking and understanding, we literally don't know what we're talking about. It's premature in the extreme to point at something that behaves so much like we do ourselves and claim that whatever it's doing, it's not "understanding" anything.
    - zlg_codes 3y ago
      We've studied human behavior enough to understand that there are differences between animals in the level of cognition and awareness they (outwardly) exhibit. Are we not generally good at detecting when someone understands us? Perhaps it's because understanding has actual meaning. If you communicate to me that you hit your head and feel like shit, I not only understand that you experienced an unsatisfactory situation, I'm capable of empathy -- understanding not only WHAT happened, but HOW it feels -- and offering consolation or high fives or whatever. A LLM has an understanding of what common responses were in the past, and repeats them. Statistical models may mimic a process we use in our thinking, but it is not the entirety of our thinking. Just like computers are limited to the programmers that code their behavior, LLMs are limited to the quality of the data corpus fed to them. A human, you can correct in real time and they'll (try to) internalize that information in future interactions. Not so with LLMs. By all means, tell us how statistically weighted answers to "what's the next word" correlates to understanding.

ethbr1 3y ago

That's not entirely accurate. LLMs encode some level of understanding of their training set. Whether that's sufficient for a specific purpose, or sufficiently comprehensive to generate side effects, is an open question. * Caveat: with regards to introspection, this also assumes it's not specifically guarded against and opaquely lying.

ekianjo 3y ago

> plausible sentence generators, they don't "understand"* their runtime environment Exactly like humans dont understand how their brain works

zlg_codes 3y ago
We've put an awfully lot of effort into figuring that out, and have some answers. Much of the problems in exploring the brain are ethical because people tend to die or suffer greatly if we experiment on them. Unlike LLMs, which are built by humans and have literal source code and manuals and SOPs and shit. Their very "body" is a well-documented digital machine. An LLM trying to figure itself out has MUCH less trouble than a human figuring itself out.

CamperBob2 3y ago

If you can watch the video demo of this release, or for that matter the Attenborough video, and still claim that these things lack any form of "understanding," then your imagination is either a lot weaker than mine, or a lot stronger. Behavior indistinguishable from understanding is understanding. Sorry, but that's how it's going to turn out to work.

zlg_codes 3y ago
Have you considered that mankind simply trained itself on the wrong criteria on detecting understanding? Why are people so eager to believe that electric rocks can think?
- CamperBob2 3y ago
  Why are people so eager to believe that people can? When it comes to the definitions of concepts like sentience, consciousness, thinking and understanding, we literally don't know what we're talking about. It's premature in the extreme to point at something that behaves so much like we do ourselves and claim that whatever it's doing, it's not "understanding" anything.
  - zlg_codes 3y ago
    We've studied human behavior enough to understand that there are differences between animals in the level of cognition and awareness they (outwardly) exhibit. Are we not generally good at detecting when someone understands us? Perhaps it's because understanding has actual meaning. If you communicate to me that you hit your head and feel like shit, I not only understand that you experienced an unsatisfactory situation, I'm capable of empathy -- understanding not only WHAT happened, but HOW it feels -- and offering consolation or high fives or whatever. A LLM has an understanding of what common responses were in the past, and repeats them. Statistical models may mimic a process we use in our thinking, but it is not the entirety of our thinking. Just like computers are limited to the programmers that code their behavior, LLMs are limited to the quality of the data corpus fed to them. A human, you can correct in real time and they'll (try to) internalize that information in future interactions. Not so with LLMs. By all means, tell us how statistically weighted answers to "what's the next word" correlates to understanding.