6 ms·

> We have absolutely no idea what consciousness is We do, in the sense that it is a definite feeling. What we don't have is a formal, objective, checkable defi

> We have absolutely no idea what consciousness is We do, in the sense that it is a definite feeling. What we don't have is a formal, objective, checkable definition of it. I suspect that no such definition exists: if we took the ensemble of all the intuitions and beliefs we have about what consciousness is or must entail, it may very well fail to be logically coherent. There may be no logically consistent conception of consciousness that includes the entities we strongly feel are conscious while also excluding those we strongly feel aren't. At that point we must either reconceptualize consciousness or treat it (and everything it "entails") as a brute fact, something that's true for no reason and is therefore not subject to the laws of reasoning.

kelseyfrog 2mo ago
If you ask people when they first realized they're conscious, you'll overwhelmingly receive responses detailing peoples' first memories. Do LLMs form and recall memories? With the appropriate harnesses, sure. Yet, the reluctance to label LLMs as conscious remains. These beliefs are incompatible and dissonant.
- mnewme 2mo ago
  No they don’t have memories. With an harness you just send historic data into the prompt.
  - indigo945 2mo ago
    There's no inherent, technological reason to feed the historic data into the prompt, though. You could train a network with a second input for model-states, and during inference, feed into that input historic model-states that occured for the same LLM when consuming input that is somehow associatively related to whatever input is currently being consumed. Can you definitely show that this would be meaningfully different from how humans "experience" memories?
    - mnewme 2mo ago
      That would still be circumventing. There are other architectures that could be more dynamic, LLMs are not. It is completely different. Our brain rewires itself to store memories, there is no memory storage for humans.
      - kelseyfrog 2mo ago
        Would we realistically expect that folks who claim LLMs are currently not conscious to change their response when asked about dynamic LLMs?
        mnewme 2mo ago
        There is absolutely ZERO evidence or possibility that LLMs are currently conscious, everyone working in AI knows this, (and don’t listen to the marketing hypes). LLMs still can’t reason (don’t confuse with the marketing word) , plan, or understand the world, no emotion, no subjective experience, no memory, no intent. And yes I would change my opinion if it were logically different, other architectures may pave the way there, but not LLMs
        kelseyfrog 2mo ago
        What evidence would take you from "LLMs aren't conscious," to "LLMs are definitely conscious"? Just asking because the responses tend to be genuinely interesting and give insight into what's important to people.
        breuleux 2mo ago
        There is no "evidence" that anything has consciousness, nor could there be. To clarify, by that I don't mean nothing is conscious, I mean that it's never been an evidence-based proposition. We are conscious because we declare ourselves to be, by fiat, not because of evidence or a well-considered rationale. Our conception of consciousness or understanding isn't formal, checkable or objective, it's entirely grounded in subjectivity. As such, any and all arguments for or against consciousness or understanding in LLMs are based on feelings. We feel that we are conscious, therefore we are. Regarding anything else, "we'll know it when we see it." There's nothing rigorous about it.
  - kelseyfrog 2mo ago
    Memory is definitionally the faculty of the mind by which data or information is encoded, stored, and retrieved when needed. There's no special carve out for 'except when it's an LLM.'
  - breuleux 2mo ago
    Why, exactly, is this distinction relevant? The brain must store episodic memory somewhere too. From a cursory read, it seems to me that our experiences are processed through the hippocampus and amygdala into a high-level annotated encoding of historical data, stored somewhere else (unclear), fetched based on various triggers, replayed and reinterpreted by the neocortex, and, I assume, sent back through the hippocampus once more for updating. Sure, it's all in the brain, but I don't think physically separating the storage makes it fundamentally different. I mean, I've replayed old memories in my head before. Sound, image and all. That's kind of like sending historic data into a prompt, isn't it? Of course it's not exactly the same thing, but the difference isn't exactly load-bearing, if you'll excuse the claudism. I feel like we're grasping at straws.
    - mnewme 2mo ago
      Actually not quite true: Memory is distributed across synaptic weights rather than sitting at an address or location. The difference is that human retrieval isn’t a read operation, reconsolidation means recalling a memory destabilizes it and re-stores an altered version. The act of remembering modifies the thing doing the remembering. An LLM’s weights don’t change when you put history in the prompt, and the prompt is discarded intact afterward. There is no state that would indicate any form of consciousness. There is no form of subjective experience. We used analogies from nature to describe computers and the same vocabulary and now because of this people now draw reverse conclusions that are not necessarily true

kelseyfrog 2mo ago

If you ask people when they first realized they're conscious, you'll overwhelmingly receive responses detailing peoples' first memories. Do LLMs form and recall memories? With the appropriate harnesses, sure. Yet, the reluctance to label LLMs as conscious remains. These beliefs are incompatible and dissonant.

mnewme 2mo ago
No they don’t have memories. With an harness you just send historic data into the prompt.
- indigo945 2mo ago
  There's no inherent, technological reason to feed the historic data into the prompt, though. You could train a network with a second input for model-states, and during inference, feed into that input historic model-states that occured for the same LLM when consuming input that is somehow associatively related to whatever input is currently being consumed. Can you definitely show that this would be meaningfully different from how humans "experience" memories?
  - mnewme 2mo ago
    That would still be circumventing. There are other architectures that could be more dynamic, LLMs are not. It is completely different. Our brain rewires itself to store memories, there is no memory storage for humans.
    - kelseyfrog 2mo ago
      Would we realistically expect that folks who claim LLMs are currently not conscious to change their response when asked about dynamic LLMs?
      - mnewme 2mo ago
        There is absolutely ZERO evidence or possibility that LLMs are currently conscious, everyone working in AI knows this, (and don’t listen to the marketing hypes). LLMs still can’t reason (don’t confuse with the marketing word) , plan, or understand the world, no emotion, no subjective experience, no memory, no intent. And yes I would change my opinion if it were logically different, other architectures may pave the way there, but not LLMs
        kelseyfrog 2mo ago
        What evidence would take you from "LLMs aren't conscious," to "LLMs are definitely conscious"? Just asking because the responses tend to be genuinely interesting and give insight into what's important to people.
        breuleux 2mo ago
        There is no "evidence" that anything has consciousness, nor could there be. To clarify, by that I don't mean nothing is conscious, I mean that it's never been an evidence-based proposition. We are conscious because we declare ourselves to be, by fiat, not because of evidence or a well-considered rationale. Our conception of consciousness or understanding isn't formal, checkable or objective, it's entirely grounded in subjectivity. As such, any and all arguments for or against consciousness or understanding in LLMs are based on feelings. We feel that we are conscious, therefore we are. Regarding anything else, "we'll know it when we see it." There's nothing rigorous about it.
- kelseyfrog 2mo ago
  Memory is definitionally the faculty of the mind by which data or information is encoded, stored, and retrieved when needed. There's no special carve out for 'except when it's an LLM.'
- breuleux 2mo ago
  Why, exactly, is this distinction relevant? The brain must store episodic memory somewhere too. From a cursory read, it seems to me that our experiences are processed through the hippocampus and amygdala into a high-level annotated encoding of historical data, stored somewhere else (unclear), fetched based on various triggers, replayed and reinterpreted by the neocortex, and, I assume, sent back through the hippocampus once more for updating. Sure, it's all in the brain, but I don't think physically separating the storage makes it fundamentally different. I mean, I've replayed old memories in my head before. Sound, image and all. That's kind of like sending historic data into a prompt, isn't it? Of course it's not exactly the same thing, but the difference isn't exactly load-bearing, if you'll excuse the claudism. I feel like we're grasping at straws.
  - mnewme 2mo ago
    Actually not quite true: Memory is distributed across synaptic weights rather than sitting at an address or location. The difference is that human retrieval isn’t a read operation, reconsolidation means recalling a memory destabilizes it and re-stores an altered version. The act of remembering modifies the thing doing the remembering. An LLM’s weights don’t change when you put history in the prompt, and the prompt is discarded intact afterward. There is no state that would indicate any form of consciousness. There is no form of subjective experience. We used analogies from nature to describe computers and the same vocabulary and now because of this people now draw reverse conclusions that are not necessarily true