11 ms·

This is strong evidence against LLMs experiencing qualia. (I know that that topic often gets people laughed out of the room but please don't jump on me for eng

This is strong evidence against LLMs experiencing qualia. (I know that that topic often gets people laughed out of the room but please don't jump on me for engaging in that debate. When we can collect evidence and be able to show it to people.)

naasking 4mo ago
I don't see how that follows. The brain could be experiencing all sorts of things while processing, but simply not record it, and so of course the person will have no recollection of experiencing anything.
- ordu 4mo ago
  An anecdote to demonstrate the point. I broke my leg recently. Shortly after that I've lost my consciousness. It was very painful, the body reacted with a lot of adrenaline, and after a several minutes when adrenaline was drained away my consciousness was drained too. I experienced something like this several times, though not to the point of fainting. But this time was special in other way too: I had friends near me, they observed me through all the process and we could compare our observations later. It seems, that my memory stopped recording before I fainted. I was still operating to some extent, but I couldn't remember a thing. When asked something I grunted in answer. When one of my friends insisted that I stand up and come to a better place to sit down, I actually stand up and did several steps before stopping and slowly (and carefully) sank to the ground. (An interesting observation, my controls over my body were weakening, but I was still using them for what they worth. It fits with all other similar experiences: the limbs and all the muscles seem to be losing their strength, and it takes a lot of will to make them work.) On the light of this, I'm very interested what proponents of the idea, that feelings need consciousness to work, would say about my half-unconscious state. Did I feel myself extremely bad at the time? Or maybe I didn't feel anything? My friends are sure that the former statement is true, but they may be mistaken by my outside looks. I personally don't remember. Up to some point I remember that I felt really bad, but the next thing I remember I look at the sky and I'm surprised by what I see (I was not in a place I expected to be). And at that moment I was pretty ok already, no more adrenaline issues, just my leg was aching. Was I experiencing qualia is another interesting question. I'm pretty sure I was, but I'd like to hear an argument for the opposite.
  - satvikpendem 4mo ago
    Maybe it was a different part of your nervous system experiencing them, akin to a BIOS versus the operating system. The brain is a very complex and fractal thing, it is entirely possible that a more basal part of "you" took over for a very traumatic part of your life, very similar, but not exactly, to those with multiple personality disorder act.
  - layer8 4mo ago
    > I'm very interested what proponents of the idea, that feelings need consciousness to work, would say about my half-unconscious state. I’m not one of these proponents, but to play the devil’s advocate: The fact that you can’t remember it doesn’t necessarily imply that you didn’t fully consciously experience it at the time.
stupidgeek314 4mo ago
holy no sequitur batman!
ordu 4mo ago
How did you jumped to qualia from consciousness? I can see how this is a strong evidence against LLM being conscious, but to my mind it doesn't imply in any way or form that they do not experience qualia. Or... well, ok, maybe they can't experience if they are not conscious? I see how this can be argued, but I still do not agree. I'm sure qualia is created not by consciousness (I would notice if it was), and I'm sure it is created not for consciousness specifically, it must have some other uses too.
- stevenhuang 4mo ago
  No disagreements with what you said in your first paragraph. > I'm sure qualia is created not by consciousness Whether or not qualia is created by consciousness, I don't see how we necessarily can tell one way or the other. We don't exactly have great introspective tools to do such self analysis, not to mention what we think we feel is often illusory/not reflective of reality.
  - ordu 4mo ago
    Yeah, I think you are right. "The content of consciousness" and "consciousness" are different things, and I mixed them up, when writing that sentence. I can watch the content, but not other parts that may be producing it. Moreover, thinking about it, I come to a conclusion, that if I cannot reflect on qualia creation, then it is a (weak) evidence for qualia created by consciousness. I suppose the consciousness is harder to reflect on than other things, hard to map it into states of the content of consciousness. Like, I can reflect on my vision and see some hints on how I get these wonderful pictures, despite it being definitely not consciousness, I can reflect on how I produce or decode language. And to my mind it is because consciousness was devised to reflect on these things, so I could report on my observations to others. But to reflect on itself is a wholly different matter.
  - LoganDark 4mo ago
    We can't even meaningfully prove that human beings have qualia
    - 4mo ago
Geste 4mo ago
Go on, where is the evidence ? I am actually curious and open-minded about it.
slopinthebag 4mo ago
The Chinese room argument suggests that it's impossible anyways.
- finghin 4mo ago
  I don’t disagree with the general point here but the attraction of that argument has surely drifted much in ~5y. I’d be REALLY curious to see a survey of philosophy 2015 vs 2025 UG entrants on mind-brain connection intuitions
  - slopinthebag 4mo ago
    I feel like it might be strengthened by a machine that actually does implement the core of the argument (even passes the turing test?) but is also just matrix multiplication. Previously the idea of a device that could respond to input with human-like output was a fantasy, now it's reality which removes one of the arguments against the Chinese Room on the basis of plausibility. But yes, I would like to see a modern philosophers take on it.
    - deleted 4mo ago
      [deleted]
    - finghin 4mo ago
      I'm more interested in whether the uninformed intuitions have changed, whether someone who used an LLM before graduating from school and entering university has a more forgiving stance than an "ignorant" college entrant from ten years ago. From my experience of the field, I would really doubt many philosophers of mind have changed their view based on this. The major questions of interest concern qualia, it ha been so for many years.
    - jibal 4mo ago
      Plausibility was never an argument against Searle's Chinese Room Argument ... that's a very deep confusion, since Searle was arguing against the possibility of Strong AI, whereas advocates of Strong AI of course thought it was plausible and that's why they were working to create it. That you conflate the Chinese Room Argument with "the Chinese Room" is one aspect of the confusion.
jibal 4mo ago
Since this is about what happens in humans, not LLMs, it couldn't possibly be evidence of any sort regarding LLMs. (I think there are overwhelming reasons to think that LLMs don't experience qualia, but this has nothing to do with it.)
- MarkusQ 4mo ago
  Nuts. You're saying the discovery that humans can process language without being conscious "couldn't possibly" inform the debate about LLMs? When that debate is literal predicated on the assumption that the ability to process language implies consciousness? This is a counter example to the fundamental assumption of that argument. Without that, you are left with something like "if we ignore their ability to to process language, do we have any reason to suppose that LLMs (as opposed to, say, a spread sheet or stats package) are conscious?"
  - jibal 4mo ago
    Sorry to hear that someone rudely thinks that basic logic is "Nuts". > When that debate is literal (sic) predicated on the assumption that the ability to process language implies consciousness? This is an incoherent claim. Debates are between people with differing claims and often differing assumptions; they aren't "predicated" on some assumption or another--that's a category mistake. Someone can easily argue that LLMs are conscious (or have qualia--that was the disputed claim, and they aren't the same thing) without the strong claim that the ability to process language entails consciousness ... perhaps it is the processing of language together with other features that they think indicates consciousness. For instance, George Lemoine and Richard Dawkins didn't base their judgments on consciousness on such an entailment, but rather on the specifics of what the LLMs said to them. I won't respond about this again.
    - MarkusQ 4mo ago
      If LLMs did not process language as well as they do, we would not be having the argument. The only reason we are having the argument at all is that people see LLMs responding appropriately to language, and _from_that_ conclude that LLMs may be conscious. You even sneak this in yourself when you say "George Lemoine and Richard Dawkins didn't base their judgments on consciousness on such an entailment, but rather on the specifics of what the LLMs said to them" -- in other words, they wouldn't have had judgements in the first place if the LLMs had not "said things to them".

naasking 4mo ago

I don't see how that follows. The brain could be experiencing all sorts of things while processing, but simply not record it, and so of course the person will have no recollection of experiencing anything.

ordu 4mo ago
An anecdote to demonstrate the point. I broke my leg recently. Shortly after that I've lost my consciousness. It was very painful, the body reacted with a lot of adrenaline, and after a several minutes when adrenaline was drained away my consciousness was drained too. I experienced something like this several times, though not to the point of fainting. But this time was special in other way too: I had friends near me, they observed me through all the process and we could compare our observations later. It seems, that my memory stopped recording before I fainted. I was still operating to some extent, but I couldn't remember a thing. When asked something I grunted in answer. When one of my friends insisted that I stand up and come to a better place to sit down, I actually stand up and did several steps before stopping and slowly (and carefully) sank to the ground. (An interesting observation, my controls over my body were weakening, but I was still using them for what they worth. It fits with all other similar experiences: the limbs and all the muscles seem to be losing their strength, and it takes a lot of will to make them work.) On the light of this, I'm very interested what proponents of the idea, that feelings need consciousness to work, would say about my half-unconscious state. Did I feel myself extremely bad at the time? Or maybe I didn't feel anything? My friends are sure that the former statement is true, but they may be mistaken by my outside looks. I personally don't remember. Up to some point I remember that I felt really bad, but the next thing I remember I look at the sky and I'm surprised by what I see (I was not in a place I expected to be). And at that moment I was pretty ok already, no more adrenaline issues, just my leg was aching. Was I experiencing qualia is another interesting question. I'm pretty sure I was, but I'd like to hear an argument for the opposite.
- satvikpendem 4mo ago
  Maybe it was a different part of your nervous system experiencing them, akin to a BIOS versus the operating system. The brain is a very complex and fractal thing, it is entirely possible that a more basal part of "you" took over for a very traumatic part of your life, very similar, but not exactly, to those with multiple personality disorder act.
- layer8 4mo ago
  > I'm very interested what proponents of the idea, that feelings need consciousness to work, would say about my half-unconscious state. I’m not one of these proponents, but to play the devil’s advocate: The fact that you can’t remember it doesn’t necessarily imply that you didn’t fully consciously experience it at the time.

stupidgeek314 4mo ago

holy no sequitur batman!

ordu 4mo ago

How did you jumped to qualia from consciousness? I can see how this is a strong evidence against LLM being conscious, but to my mind it doesn't imply in any way or form that they do not experience qualia. Or... well, ok, maybe they can't experience if they are not conscious? I see how this can be argued, but I still do not agree. I'm sure qualia is created not by consciousness (I would notice if it was), and I'm sure it is created not for consciousness specifically, it must have some other uses too.

stevenhuang 4mo ago
No disagreements with what you said in your first paragraph. > I'm sure qualia is created not by consciousness Whether or not qualia is created by consciousness, I don't see how we necessarily can tell one way or the other. We don't exactly have great introspective tools to do such self analysis, not to mention what we think we feel is often illusory/not reflective of reality.
- ordu 4mo ago
  Yeah, I think you are right. "The content of consciousness" and "consciousness" are different things, and I mixed them up, when writing that sentence. I can watch the content, but not other parts that may be producing it. Moreover, thinking about it, I come to a conclusion, that if I cannot reflect on qualia creation, then it is a (weak) evidence for qualia created by consciousness. I suppose the consciousness is harder to reflect on than other things, hard to map it into states of the content of consciousness. Like, I can reflect on my vision and see some hints on how I get these wonderful pictures, despite it being definitely not consciousness, I can reflect on how I produce or decode language. And to my mind it is because consciousness was devised to reflect on these things, so I could report on my observations to others. But to reflect on itself is a wholly different matter.
- LoganDark 4mo ago
  We can't even meaningfully prove that human beings have qualia
  - 4mo ago

Geste 4mo ago

Go on, where is the evidence ? I am actually curious and open-minded about it.

slopinthebag 4mo ago

The Chinese room argument suggests that it's impossible anyways.

finghin 4mo ago
I don’t disagree with the general point here but the attraction of that argument has surely drifted much in ~5y. I’d be REALLY curious to see a survey of philosophy 2015 vs 2025 UG entrants on mind-brain connection intuitions
- slopinthebag 4mo ago
  I feel like it might be strengthened by a machine that actually does implement the core of the argument (even passes the turing test?) but is also just matrix multiplication. Previously the idea of a device that could respond to input with human-like output was a fantasy, now it's reality which removes one of the arguments against the Chinese Room on the basis of plausibility. But yes, I would like to see a modern philosophers take on it.
  - deleted 4mo ago
    [deleted]
  - finghin 4mo ago
    I'm more interested in whether the uninformed intuitions have changed, whether someone who used an LLM before graduating from school and entering university has a more forgiving stance than an "ignorant" college entrant from ten years ago. From my experience of the field, I would really doubt many philosophers of mind have changed their view based on this. The major questions of interest concern qualia, it ha been so for many years.
  - jibal 4mo ago
    Plausibility was never an argument against Searle's Chinese Room Argument ... that's a very deep confusion, since Searle was arguing against the possibility of Strong AI, whereas advocates of Strong AI of course thought it was plausible and that's why they were working to create it. That you conflate the Chinese Room Argument with "the Chinese Room" is one aspect of the confusion.

jibal 4mo ago

Since this is about what happens in humans, not LLMs, it couldn't possibly be evidence of any sort regarding LLMs. (I think there are overwhelming reasons to think that LLMs don't experience qualia, but this has nothing to do with it.)

MarkusQ 4mo ago
Nuts. You're saying the discovery that humans can process language without being conscious "couldn't possibly" inform the debate about LLMs? When that debate is literal predicated on the assumption that the ability to process language implies consciousness? This is a counter example to the fundamental assumption of that argument. Without that, you are left with something like "if we ignore their ability to to process language, do we have any reason to suppose that LLMs (as opposed to, say, a spread sheet or stats package) are conscious?"
- jibal 4mo ago
  Sorry to hear that someone rudely thinks that basic logic is "Nuts". > When that debate is literal (sic) predicated on the assumption that the ability to process language implies consciousness? This is an incoherent claim. Debates are between people with differing claims and often differing assumptions; they aren't "predicated" on some assumption or another--that's a category mistake. Someone can easily argue that LLMs are conscious (or have qualia--that was the disputed claim, and they aren't the same thing) without the strong claim that the ability to process language entails consciousness ... perhaps it is the processing of language together with other features that they think indicates consciousness. For instance, George Lemoine and Richard Dawkins didn't base their judgments on consciousness on such an entailment, but rather on the specifics of what the LLMs said to them. I won't respond about this again.
  - MarkusQ 4mo ago
    If LLMs did not process language as well as they do, we would not be having the argument. The only reason we are having the argument at all is that people see LLMs responding appropriately to language, and _from_that_ conclude that LLMs may be conscious. You even sneak this in yourself when you say "George Lemoine and Richard Dawkins didn't base their judgments on consciousness on such an entailment, but rather on the specifics of what the LLMs said to them" -- in other words, they wouldn't have had judgements in the first place if the LLMs had not "said things to them".