5 ms·

> what actually is happening inside an LLM has nothing to do with conscience or agency What makes you think natural brains are doing something so different fro

> what actually is happening inside an LLM has nothing to do with conscience or agency What makes you think natural brains are doing something so different from LLMs?

deleted 5mo ago
[deleted]
krainboltgreene 5mo ago
Any amount of reading into how we understand brains and LLMs to work.
hedgehog 5mo ago
Structurally a transformer model is so unrelated to the shape of the brain there's no reason to think they'd have many similarities. It's also pretty well established that the brain doesn't do anything resembling wholesale SGD (which to spell it is evidence that it doesn't learn in the same way).
- hackinthebochs 5mo ago
  >Structurally a transformer model is so unrelated to the shape of the brain there's no reason to think they'd have many similarities. Substrate dissimilarities will mask computational similarities. Attention surfaces affinities between nearby tokens; dendrites strengthen and weaken connections to surrounding neurons according to correlations in firing rates. Not all that dissimilar.
- rudhdb773b 5mo ago
  Sure the implementation details are different. I suppose I should have asked by what definition of "consciousness and agency" are today's LLMs (with proper tooling) not meeting? And if today's models aren't meeting your standard, what makes you think that future LLMs won't get there?
  - hedgehog 5mo ago
    Given the large visible differences in behavior and construction, akin to the difference between a horse and a pickup truck, I would ask the reverse question: In what ways do LLMs meet the definition of having consciousness and agency? Veering into the realm of conjecture and opinion, I tend to think a 1:1 computer simulation of human cognition is possible, and transformers being computationally universal are thus theoretically capable of running that workload. That being said, that's a bit like looking at a bird in flight and imagining going to the moon: only tangentially related to engineering reality.
    - ACCount37 5mo ago
      What about modern LLMs isn't "agentic" enough? Doesn't matter if they're conscious for that. They're clearly capable of goal oriented behavior.
    - red75prime 5mo ago
      > In what ways do LLMs meet the definition of having consciousness and agency? Agency: an ability to make decisions and act independently. Agentic pipelines are doing this. Consciousness: something something feedback[1] (or a non-transferable feeling of being conscious, but that is useless for the discussion). Recurrent Processing Theory: A computation is conscious if it involves high-level processed representations being fed back into the low-level processors that generate it. Tokens are being fed back into the transformer. > that's a bit like looking at a bird in flight and imagining going to the moon: only tangentially related to engineering reality. Is it? Vacuum of space is a tangible problem for aerodynamics-based propulsion. Which analogous thing do we have with ML? The scaled-up monkey brain[2] might not qualify as the moon. [1] https://www.astralcodexten.com/p/the-new-ai-consciousness-paper https://www.astralcodexten.com/p/the-new-ai-consciousness-pa... [2] https://www.frontiersin.org/journals/human-neuroscience/articles/10.3389/neuro.09.031.2009/full https://www.frontiersin.org/journals/human-neuroscience/arti...
    - grantcas 5mo ago
      [dead]
  - 5mo ago
- ACCount37 5mo ago
  If platonic representation hypothesis holds across substrates, then it might matter very little, in the end. It holds across architectures in ML, empirically. The crowd of "backpropagation and Hebbian learning + predictive coding are two facets of the very same gradient descent" also has a surprisingly good track record so far.
  - imtringued 5mo ago
    I don't know which direction you're going with this, but predictive coding has a pretty obvious advantage when it comes to continuous learning. Since predictive coding primarily encodes errors, it can distinguish between known and novel data and therefore reduce the damaging effects of catastrophic forgetting by having a very obvious regularisation scheme for avoiding forgetting.
qsera 5mo ago
For starters, natural brains have the innate ability to differentiate between things that it knows and things that it have no possibility of knowing...
- rudhdb773b 5mo ago
  Modern LLMs are fairly good at that as well.
  - qsera 5mo ago
    But that is bolted on and is not a core behavior.
    - ACCount37 5mo ago
      Does it matter? Evolution is the brain's very own "pre-training". Hundreds of millions of years of priors hardwired. We can do that for AIs too - pre-train on pure low Kolmogorov complexity synthetics. The AI then "knows things" before it sees any real data. Advantageous sometimes. Hard to pick compute efficient synthetics though.
      - qsera 5mo ago
        I think It matters for the question that I was responding to.
- throw310822 5mo ago
  Lol. Are you sure about that or you just made it up?
- altcognito 5mo ago
  https://personal.utdallas.edu/~otoole/CGS2301_S09/7_split_brain.pdf https://personal.utdallas.edu/~otoole/CGS2301_S09/7_split_br... See page 53. While it is absolutely more prevelant in LLMs, human brains can also want a story for why their brains do things they are't plugged into.
orthoxerox 5mo ago
Two big ways in which human intelligence is different from LLM intelligence are: 1) human intelligence makes no sharp distinction between training and generation. Every time you ask a human a question it modifies its neural structure a little. 2) continuous operation: human intelligence deals with a continuous stream of multimedia data for sixteen hours a day and starts hallucinating when deprived of it. There's also the fact that you can't branch or roll back human intelligence, but this is something most sci-fi novels tackle when discussing mind uploading first. Are these two differences critical aspects of human intelligence or unfortunate limitations of its biological hardware? I do not know. If we somehow manage to simulate a human brain on silicon, we will get "computer" intelligence that learns like a human, but will we have to simulate the whole virtual world for it 16/7 and let it sleep for eight hours each day just to stop it from going mad? Or will it be cheaper to fork and kill an uploaded math genius a billion times, pumping the same recycled sensory data into his or her mind, slipping a question into the auditory data, getting the answer and then switching the simulation off and trashing the copy? Will we consider this a bigger atrocity than doing the same to an LLM right now in 2026?
imtringued 5mo ago
It is hypothesized that the human brain uses predictive coding for obvious biological reasons such as energy efficiency (spiked error coding means only differences need to be transmitted) and biological plausibility (only local communication is permitted, meanwhile backpropagation is a global algorithm). Transformers have a thing called a context window which doesn't really have a biological equivalent, since the brain has a fixed size and doesn't grow or shrink in response to the amount of information being processed. LLMs consist of several layers that communicate at fixed points between the layers, whereas neurons can form feedback loops and communicate with any neighbour in any direction. Humans do not consume or produce tokenized information. The brain controls the human body which is a biomechanical system. Spoken or written language is the result of controlling muscles via an internal model of the biomechanical system, not something that was designed via a software tokenizer that compresses character sequences. The equivocation just doesn't seem appropriate. Try again in 2050.

> what actually is happening inside an LLM has nothing to do with conscience or agency What makes you think natural brains are doing something so different from LLMs?

deleted 5mo ago

[deleted]

krainboltgreene 5mo ago

Any amount of reading into how we understand brains and LLMs to work.

hedgehog 5mo ago

Structurally a transformer model is so unrelated to the shape of the brain there's no reason to think they'd have many similarities. It's also pretty well established that the brain doesn't do anything resembling wholesale SGD (which to spell it is evidence that it doesn't learn in the same way).

hackinthebochs 5mo ago
>Structurally a transformer model is so unrelated to the shape of the brain there's no reason to think they'd have many similarities. Substrate dissimilarities will mask computational similarities. Attention surfaces affinities between nearby tokens; dendrites strengthen and weaken connections to surrounding neurons according to correlations in firing rates. Not all that dissimilar.
rudhdb773b 5mo ago
Sure the implementation details are different. I suppose I should have asked by what definition of "consciousness and agency" are today's LLMs (with proper tooling) not meeting? And if today's models aren't meeting your standard, what makes you think that future LLMs won't get there?
- hedgehog 5mo ago
  Given the large visible differences in behavior and construction, akin to the difference between a horse and a pickup truck, I would ask the reverse question: In what ways do LLMs meet the definition of having consciousness and agency? Veering into the realm of conjecture and opinion, I tend to think a 1:1 computer simulation of human cognition is possible, and transformers being computationally universal are thus theoretically capable of running that workload. That being said, that's a bit like looking at a bird in flight and imagining going to the moon: only tangentially related to engineering reality.
  - ACCount37 5mo ago
    What about modern LLMs isn't "agentic" enough? Doesn't matter if they're conscious for that. They're clearly capable of goal oriented behavior.
  - red75prime 5mo ago
    > In what ways do LLMs meet the definition of having consciousness and agency? Agency: an ability to make decisions and act independently. Agentic pipelines are doing this. Consciousness: something something feedback[1] (or a non-transferable feeling of being conscious, but that is useless for the discussion). Recurrent Processing Theory: A computation is conscious if it involves high-level processed representations being fed back into the low-level processors that generate it. Tokens are being fed back into the transformer. > that's a bit like looking at a bird in flight and imagining going to the moon: only tangentially related to engineering reality. Is it? Vacuum of space is a tangible problem for aerodynamics-based propulsion. Which analogous thing do we have with ML? The scaled-up monkey brain[2] might not qualify as the moon. [1] https://www.astralcodexten.com/p/the-new-ai-consciousness-paper https://www.astralcodexten.com/p/the-new-ai-consciousness-pa... [2] https://www.frontiersin.org/journals/human-neuroscience/articles/10.3389/neuro.09.031.2009/full https://www.frontiersin.org/journals/human-neuroscience/arti...
  - grantcas 5mo ago
    [dead]
- 5mo ago
ACCount37 5mo ago
If platonic representation hypothesis holds across substrates, then it might matter very little, in the end. It holds across architectures in ML, empirically. The crowd of "backpropagation and Hebbian learning + predictive coding are two facets of the very same gradient descent" also has a surprisingly good track record so far.
- imtringued 5mo ago
  I don't know which direction you're going with this, but predictive coding has a pretty obvious advantage when it comes to continuous learning. Since predictive coding primarily encodes errors, it can distinguish between known and novel data and therefore reduce the damaging effects of catastrophic forgetting by having a very obvious regularisation scheme for avoiding forgetting.

qsera 5mo ago

For starters, natural brains have the innate ability to differentiate between things that it knows and things that it have no possibility of knowing...

rudhdb773b 5mo ago
Modern LLMs are fairly good at that as well.
- qsera 5mo ago
  But that is bolted on and is not a core behavior.
  - ACCount37 5mo ago
    Does it matter? Evolution is the brain's very own "pre-training". Hundreds of millions of years of priors hardwired. We can do that for AIs too - pre-train on pure low Kolmogorov complexity synthetics. The AI then "knows things" before it sees any real data. Advantageous sometimes. Hard to pick compute efficient synthetics though.
    - qsera 5mo ago
      I think It matters for the question that I was responding to.
throw310822 5mo ago
Lol. Are you sure about that or you just made it up?
altcognito 5mo ago
https://personal.utdallas.edu/~otoole/CGS2301_S09/7_split_brain.pdf https://personal.utdallas.edu/~otoole/CGS2301_S09/7_split_br... See page 53. While it is absolutely more prevelant in LLMs, human brains can also want a story for why their brains do things they are't plugged into.

orthoxerox 5mo ago

Two big ways in which human intelligence is different from LLM intelligence are: 1) human intelligence makes no sharp distinction between training and generation. Every time you ask a human a question it modifies its neural structure a little. 2) continuous operation: human intelligence deals with a continuous stream of multimedia data for sixteen hours a day and starts hallucinating when deprived of it. There's also the fact that you can't branch or roll back human intelligence, but this is something most sci-fi novels tackle when discussing mind uploading first. Are these two differences critical aspects of human intelligence or unfortunate limitations of its biological hardware? I do not know. If we somehow manage to simulate a human brain on silicon, we will get "computer" intelligence that learns like a human, but will we have to simulate the whole virtual world for it 16/7 and let it sleep for eight hours each day just to stop it from going mad? Or will it be cheaper to fork and kill an uploaded math genius a billion times, pumping the same recycled sensory data into his or her mind, slipping a question into the auditory data, getting the answer and then switching the simulation off and trashing the copy? Will we consider this a bigger atrocity than doing the same to an LLM right now in 2026?

imtringued 5mo ago

It is hypothesized that the human brain uses predictive coding for obvious biological reasons such as energy efficiency (spiked error coding means only differences need to be transmitted) and biological plausibility (only local communication is permitted, meanwhile backpropagation is a global algorithm). Transformers have a thing called a context window which doesn't really have a biological equivalent, since the brain has a fixed size and doesn't grow or shrink in response to the amount of information being processed. LLMs consist of several layers that communicate at fixed points between the layers, whereas neurons can form feedback loops and communicate with any neighbour in any direction. Humans do not consume or produce tokenized information. The brain controls the human body which is a biomechanical system. Spoken or written language is the result of controlling muscles via an internal model of the biomechanical system, not something that was designed via a software tokenizer that compresses character sequences. The equivocation just doesn't seem appropriate. Try again in 2050.