4 ms·

sigh; this argument is the new Chinese Room; easily described, utterly wrong. https://www.youtube.com/watch?v=YEUclZdj_Sc https://www.youtube.com/watch?v=YEUcl

sigh; this argument is the new Chinese Room; easily described, utterly wrong. https://www.youtube.com/watch?v=YEUclZdj_Sc https://www.youtube.com/watch?v=YEUclZdj_Sc

razorbeamz 6mo ago
Next-token-prediction cannot do calculations. That is fundamental. It can produce outputs that resemble calculations. It can prompt an agent to input some numbers into a separate program that will do calculations for it and then return them as a prompt. Neither of these are calculations.
- parasubvert 6mo ago
  Humans can't do calculations either, by your definition. Only computers can.
  - datsci_est_2015 6mo ago
    Third things can exist. In other words, you’re implying a false dichotomy between “human computation” and “computer computation” and implying that LLMs must be one or the other. A pithy gotcha comment, no doubt. Edit: the implication comes from demanding that the OP’s definition must be rigorous enough to cover all models of “computation”, and by failing to do so, it means that LLMs must be more like humans than computers.
- gf000 6mo ago
  So you don't think 50T parameter neural networks can encode the logic for adding two n-bit integers for reasonably sized integers? That would be pretty sad.
  - razorbeamz 6mo ago
    They do not. The fundamental technology behind LLMs does not allow that to be the case. You are hoping that an LLM can do something that it cannot do.
    - gf000 6mo ago
      https://arxiv.org/html/2502.16763v2 https://arxiv.org/html/2502.16763v2 You are wrong. Especially that we are talking about models with 50T parameters. Can they do arbitrary computations for arbitrarily long numbers? Nope. But that's not remotely the same statement, and they can trivially call out to tools to do that in those cases.
    - int_19h 6mo ago
      You do realize that training a neural net to do addition is a beginner level exercise in ML?
gpderetta 6mo ago
After dismissing it for a long time, I have come around to the philosophical zombie argument. I do not believe that LLMs are conscious, but I also no longer believe that consciousness is a prerequisite for intelligence. I think at this point it is hard to deny that LLMs do not possess some form of intelligence (although not necessarily human-like). I think P-zombies is a fitting description.
- zeroonetwothree 6mo ago
  I don't think P-zombies can exist. There must be some perceptible difference between an intelligence w/ consciousness and one without. The only way there wouldn't be a difference is if we are mistaken about the consciousness (either both have it or neither do).
  - gpderetta 6mo ago
    > There must be some perceptible difference between an intelligence w/ consciousness and one without I think there are differences, and I think we can make good guesses, but I'm not sure we can reliably classify a P-zombie from a normal human from their behaviour with 100% accuracy..

sigh; this argument is the new Chinese Room; easily described, utterly wrong. https://www.youtube.com/watch?v=YEUclZdj_Sc https://www.youtube.com/watch?v=YEUclZdj_Sc

razorbeamz 6mo ago

Next-token-prediction cannot do calculations. That is fundamental. It can produce outputs that resemble calculations. It can prompt an agent to input some numbers into a separate program that will do calculations for it and then return them as a prompt. Neither of these are calculations.

parasubvert 6mo ago
Humans can't do calculations either, by your definition. Only computers can.
- datsci_est_2015 6mo ago
  Third things can exist. In other words, you’re implying a false dichotomy between “human computation” and “computer computation” and implying that LLMs must be one or the other. A pithy gotcha comment, no doubt. Edit: the implication comes from demanding that the OP’s definition must be rigorous enough to cover all models of “computation”, and by failing to do so, it means that LLMs must be more like humans than computers.
gf000 6mo ago
So you don't think 50T parameter neural networks can encode the logic for adding two n-bit integers for reasonably sized integers? That would be pretty sad.
- razorbeamz 6mo ago
  They do not. The fundamental technology behind LLMs does not allow that to be the case. You are hoping that an LLM can do something that it cannot do.
  - gf000 6mo ago
    https://arxiv.org/html/2502.16763v2 https://arxiv.org/html/2502.16763v2 You are wrong. Especially that we are talking about models with 50T parameters. Can they do arbitrary computations for arbitrarily long numbers? Nope. But that's not remotely the same statement, and they can trivially call out to tools to do that in those cases.
  - int_19h 6mo ago
    You do realize that training a neural net to do addition is a beginner level exercise in ML?

gpderetta 6mo ago

After dismissing it for a long time, I have come around to the philosophical zombie argument. I do not believe that LLMs are conscious, but I also no longer believe that consciousness is a prerequisite for intelligence. I think at this point it is hard to deny that LLMs do not possess some form of intelligence (although not necessarily human-like). I think P-zombies is a fitting description.

zeroonetwothree 6mo ago
I don't think P-zombies can exist. There must be some perceptible difference between an intelligence w/ consciousness and one without. The only way there wouldn't be a difference is if we are mistaken about the consciousness (either both have it or neither do).
- gpderetta 6mo ago
  > There must be some perceptible difference between an intelligence w/ consciousness and one without I think there are differences, and I think we can make good guesses, but I'm not sure we can reliably classify a P-zombie from a normal human from their behaviour with 100% accuracy..