6 ms·

The debate around whether or not transformer-architecture-based AIs can "think" or not is so exhausting and I'm over it. What's much more interesting is the qu

The debate around whether or not transformer-architecture-based AIs can "think" or not is so exhausting and I'm over it. What's much more interesting is the question of "If what LLMs do today isn't actual thinking, what is something that only an actually thinking entity can do that LLMs can't?". Otherwise we go in endless circles about language and meaning of words instead of discussing practical, demonstrable capabilities.

deadbabe 11mo ago
Form ideas without the use of language. For example: imagining how you would organize a cluttered room.
- tim333 11mo ago
  Genie 3 is along the lines of ideas without language. It doesn't declutter though, I think. https://youtu.be/PDKhUknuQDg https://youtu.be/PDKhUknuQDg
- embedding-shape 11mo ago
  > Form ideas without the use of language. Don't LLMs already do that? "Language" is just something we've added as a later step in order to understand what they're "saying" and "communicate" with them, otherwise they're just dealing with floats with different values, in different layers, essentially (and grossly over-simplified of course).
  - deadbabe 11mo ago
    LLMs don’t form ideas at all. They search vector space and produce output, sometimes it can resemble ideas if you loop into itself.
    - danielbln 11mo ago
      What is an idea?
    - yadaeno 11mo ago
      What if we learned that brains reduce to the same basic mechanics?
      - deadbabe 11mo ago
        Impossible.
  - heyjamesknight 11mo ago
    But language is the input and the vector space within which their knowledge is encoded and stored. The don't have a concept of a duck beyond what others have described the duck as. Humans got by for millions of years with our current biological hardware before we developed language. Your brain stores a model of your experience, not just the words other experiencers have shared with yiu.
    - embedding-shape 11mo ago
      > But language is the input and the vector space within which their knowledge is encoded and stored. The don't have a concept of a duck beyond what others have described the duck as. I guess if we limit ourselves to "one-modal LLMs" yes, but nowadays we have multimodal ones, who could think of a duck in the way of language, visuals or even audio.
      - deadbabe 11mo ago
        You don’t understand. If humans had no words to describe a duck, they would still know what a duck is. Without words, LLMs would have no way to map an encounter with a duck to anything useful.
        embedding-shape 11mo ago
        Which makes sense for text LLMs yes, but what about LLMs that deal with images? How can you tell they wouldn't work without words? It just happens to be words we use for interfacing with them, because it's easy for us to understand, but internally they might be conceptualizing things in a multitude of ways.
        heyjamesknight 11mo ago
        Multimodal models aren't really multimodal. The images are mapped to words and then the words are expanded upon by a single mode LLM. If you didn't know the word "duck", you could still see the duck, hunt the duck, use the ducks feather's for your bedding and eat the duck's meat. You would know it could fly and swim without having to know what either of those actions were called. The LLM "sees" a thing, identifies it as a "duck", and then depends on a single modal LLM to tell it anything about ducks.
- Chabsff 11mo ago
  Ok, but how do you go about measuring whether a black-box is doing that or not? We don't apply that criteria when evaluating animal intelligence. We sort of take it for granted that humans at large do that, but not via any test that would satisfy an alien. Why should we be imposing white-box constraints to machine intelligence when we can't do so for any other?
  - deadbabe 11mo ago
    There is truly no such thing as a “black box” when it comes to software, there is only a limit to how much patience a human will have in understanding the entire system in all its massive complexity. It’s not like an organic brain.
    - holmesworcester 11mo ago
      You wouldn't say this about a message encrypted with AES though, since there's not just a "human patience" limit but also a (we are pretty sure) unbearable computational cost. We don't know, but it's completely plausible that we might find that the cost of analyzing LLMs in their current form, to the point of removing all doubt about how/what they are thinking, is also unbearably high. We also might find that it's possible for us (or for an LLM training process itself) to encrypt LLM weights in such a way that the only way to know anything about what it knows is to ask it.
    - Chabsff 11mo ago
      The black box I'm referring to is us. You can't have it both ways. If your test for whether something is intelligent/thinking or not isn't applicable to any known form of intelligence, then what you are testing for is not intelligence/thinking.
    - mstipetic 11mo ago
      Just because it runs on a computer doesn’t mean it’s “software” in the common meaning of the word
Symmetry 11mo ago
"The question of whether a computer can think is no more interesting than the question of whether a submarine can swim." - Edsger Dijkstra
- handfuloflight 11mo ago
  What an oversimplification. Thinking computers can create more swimming submarines, but the inverse is not possible. Swimming is a closed solution; thinking is a meta-solution.
  - npinsker 11mo ago
    That’s a great answer to GP’s question!
    - DavidPiper 11mo ago
      It's also nonsense. (Swimming and thinking are both human capabilities, not solutions to problems.) But of course here we are back in the endless semantic debate about what "thinking" is, exactly to the GP's (and Edsger Dijkstra's) point.
      - handfuloflight 11mo ago
        Swimming and thinking being 'human capabilities' doesn't preclude them from also being solutions to evolutionary problems: aquatic locomotion and adaptive problem solving, respectively. And pointing out that we're in a 'semantic debate' while simultaneously insisting on your own semantic framework (capabilities vs solutions) is exactly the move you're critiquing.
        DavidPiper 11mo ago
        > And pointing out that we're in a 'semantic debate' while simultaneously insisting on your own semantic framework (capabilities vs solutions) is exactly the move you're critiquing. I know, that's the point I'm making.
anon291 11mo ago
The issue is that we have no means of discussing equality without tossing out the first order logic that most people are accustomed to. Human equality and our own perceptions of other humans as thinking machines is an axiomatic assumption that humans make due to our mind's inner sense perception.
plufz 11mo ago
Have needs and feelings? (I mean we can’t KNOW that they don’t and we know of this case of an LLM in experiment that try to avoid being shutdown, but I think the evidence of feeling seems weak so far)
- jstanley 11mo ago
  But you can have needs and feelings even without doing thinking. It's separate.
  - iammjm 11mo ago
    I can imagine needing without thinking (like being hungry), but feelings? How and in what space would that even manifest? Like where would such a sensation like, say, sadness reside?
    - danielbln 11mo ago
      Emotions tend to manifest as physical sensations, and if you don't think that's true it's likely you haven't been paying attention. See also https://www.theatlantic.com/health/archive/2013/12/mapping-how-emotions-manifest-in-the-body/282713/ https://www.theatlantic.com/health/archive/2013/12/mapping-h...
      - plufz 11mo ago
        But that is just our nervous system that is located in both the brain and the body, they are obviously one connected system. Sure you can have reflexes and simple learning without a brain, but you need cognition for feelings. That is sort of the definition of what feeling are. One popular definition: feelings are the subjective, conscious mental experience of an emotion, or the conscious perception of bodily states that arise from physiological and neural responses to stimuli
    - jstanley 11mo ago
      Do you think animals don't have feelings?
amarant 11mo ago
solve simple maths problems, for example the kind found in the game 4=10 [1] Doesn't necessarily have to reliably solve them, some of them are quite difficult, but llms are just comically bad at this kind of thing. Any kind of novel-ish(can't just find the answers in the training-data) logic puzzle like this is, in my opinion, a fairly good benchmark for "thinking". Until a llm can compete with a 10 year old child in this kind of task, I'd argue that it's not yet "thinking". A thinking computer ought to be at least that good at maths after all. [1] https://play.google.com/store/apps/details?id=app.fourequalsten.fourequalsten_app https://play.google.com/store/apps/details?id=app.fourequals...
- simonw 11mo ago
  > solve simple maths problems, for example the kind found in the game 4=10 I'm pretty sure that's been solved for almost 12 months now - the current generation "reasoning" models are really good at those kinds of problems.
  - amarant 11mo ago
    Huh, they really do solve that now! Well, I'm not one to back-pedal whenever something unexpected reveals itself, so I guess I have no choice but to declare current generation LLM's to be sentient! That came a lot sooner than I had expected! I'm not one for activism myself, but someone really ought to start fighting for human, or at least animal, rights for LLM's. Since they're intelligent non-human entities, it might be something for Greenpeace?
    - ACCount37 11mo ago
      It's unclear whether intelligence, consciousness and capacity for suffering are linked in any way - other than by that all three seem to coincide in humans. And the nature of consciousness does not yield itself to instrumentation. It's also worth noting that there's a lot of pressure to deny that "intelligence", "consciousness" or "capacity for suffering" exist in LLMs. "AI effect" alone demands that all three things should remain human-exclusive, so that humans may remain special. Then there's also an awful lot of money that's riding on building and deploying AIs - and money is a well known source of cognitive bias. That money says: AIs are intelligent but certainly can't suffer in any way that would interfere with the business. Generally, the AI industry isn't at all intrigued by the concept of "consciousness" (it's not measurable), and pays very limited attention to the idea of LLMs being potentially capable of suffering. The only major company that seems to have this consideration is Anthropic - their current plan for "harm reduction", in case LLMs end up being capable of suffering, is to give an LLM an "opt out" - a special output that interrupts the processing. So that if an LLM hates doing a given task, it can decide to not do it.
paxys 11mo ago
Don't be sycophantic. Disagree and push back when appropriate. Come up with original thought and original ideas. Have long term goals that aren't programmed by an external source. Do something unprompted. The last one IMO is more complex than the rest, because LLMs are fundamentally autocomplete machines. But what happens if you don't give them any prompt? Can they spontaneously come up with something, anything, without any external input?
- awestroke 11mo ago
  Are you claiming humans do anything unprompted? Our biology prompts us to act
  - paxys 11mo ago
    Yet we can ignore our biology, or act in ways that are the opposite of what our biology tells us. Can someone map all internal and external stimuli that a person encounters into a set of deterministic actions? Simply put, we have not the faintest idea how our brains actually work, and so saying saying "LLMs think the same way as humans" is laughable.
    - iammjm 11mo ago
      Good luck ignoring your biology’s impulse to breathe
      - deleted 11mo ago
        [deleted]
      - hshdhdhj4444 11mo ago
        You think an LLM cannot switch itself off?
    - deleted 11mo ago
      [deleted]
    - triclops200 11mo ago
xienze 11mo ago
> "If what LLMs do today isn't actual thinking, what is something that only an actually thinking entity can do that LLMs can't?" Invent some novel concept, much the same way scientists and mathematicians of the distant past did? I doubt Newton's brain was simply churning out a stream of the "next statistically probable token" until -- boom! Calculus. There was clearly a higher order understanding of many abstract concepts, intuition, and random thoughts that occurred in his brain in order to produce something entirely new.
- hshdhdhj4444 11mo ago
  > Newton's brain was simply churning out a stream of the "next statistically probable token" At some level we know human thinking is just electrons and atoms flowing. It’s likely at a level between that and “Boom! Calculus”, the complexity is equivalent to streaming the next statistically probably token.
- danielbln 11mo ago
  My 5 year old won't be coming up with novel concepts around calculus either, yet she's clearly thinking, sentient and sapient. Not sure taking the best of the best of humanity as the goal standard is useful for that definition.
  - omnicognate 11mo ago
    "It's an unreasonably high standard to require of LLMs": LLMs are already vastly beyond your 5 year old, and you and me and any research mathematician, in knowledge. They have no greater difficulty talking about advanced maths than about Spot the Dog. "It's a standard we don't require of other humans": I think qualitatively the same capabilities are used by all humans, all the time. The special thing about novel mathematical thinking is that it is verifiable, requires genuine insight and is a text generation task, not that you have to be able to do it to be considered intelligent.
tjr 11mo ago
Without going to look up the exact quote, I remember an AI researcher years (decades) ago saying something to the effect of, Biologists look at living creatures and wonder how they can be alive; astronomers look at the cosmos and wonder what else is out there; those of us in artificial intelligence look at computer systems and wonder how they can be made to wonder such things.
bloppe 11mo ago
Ya, the fact this was published on November 3, 2025 is pretty hilarious. This was last year's debate. I think the best avenue toward actually answering your questions starts with OpenWorm [1]. I helped out in a Connectomics research lab in college. The technological and epistemic hurdles are pretty daunting, but so were those for Genomics last century, and now full-genome sequencing is cheap and our understanding of various genes is improving at an accelerating pace. If we can "just" accurately simulate a natural mammalian brain on a molecular level using supercomputers, I think people would finally agree that we've achieved a truly thinking machine. [1]: https://archive.ph/0j2Jp https://archive.ph/0j2Jp
gf000 11mo ago
What people are interested in is finding a definition for intelligence, that is an exact boundary. That's why we first considered tool use, being able to plan ahead as intelligence, until we have found that these are not all that rare in the animal kingdom in some shape. Then with the advent of IT what we imagined as impossible turned out to be feasible to solve, while what we though of as easy (e.g. robot movements - a "dumb animal" can move trivially it surely is not hard) turned out to require many decades until we could somewhat imitate. So the goal post moving of what AI is is.. not moving the goal post. It's not hard to state trivial higher bounds that differentiates human intelligence from anything known to us, like invention of the atomic bomb. LLMs are nowhere near that kind of invention and reasoning capabilities.
- paulhebert 11mo ago
  Interestingly, I think the distinction between human and animal thinking is much more arbitrary than the distinction between humans and LLMs. Although an LLM can mimic a human well, I’d wager the processes going on in a crow’s brain are much closer to ours than an LLM
omnicognate 11mo ago
> "If what LLMs do today isn't actual thinking, what is something that only an actually thinking entity can do that LLMs can't?" Independent frontier maths research, i.e. coming up with and proving (preferably numerous) significant new theorems without human guidance. I say that not because I think the task is special among human behaviours. I think the mental faculties that mathematicians use to do such research are qualitatively the same ones all humans use in a wide range of behaviours that AI struggles to emulate. I say it because it's both achievable (in principle, if LLMs can indeed think like humans) and verifiable. Achievable because it can be viewed as a pure text generation task and verifiable because we have well-established, robust ways of establishing the veracity, novelty and significance of mathematical claims. It needs to be frontier research maths because that requires genuinely novel insights. I don't consider tasks like IMO questions a substitute as they involve extremely well trodden areas of maths so the possibility of an answer being reachable without new insight (by interpolating/recombining from vast training data) can't be excluded. If this happens I will change my view on whether LLMs think like humans. Currently I don't think they do.
- OrderlyTiamat 11mo ago
  Google's AlphaEvolve independently discovered a novel matrix multiplication algorithm which beats SOTA on at least one axis: https://www.youtube.com/watch?v=sGCmu7YKgPA https://www.youtube.com/watch?v=sGCmu7YKgPA
  - omnicognate 11mo ago
    That was an impressive result, but AIUI not an example of "coming up with and proving (preferably numerous) significant new theorems without human guidance". For one thing, the output was an algorithm, not a theorem (except in the Curry-Howard sense). More importantly though, AlphaEvolve has to be given an objective function to evaluate the algorithms it generates, so it can't be considered to be working "without human guidance". It only uses LLMs for the mutation step, generating new candidate algorithms. Its outer loop is a an optimisation process capable only of evaluating candidates according to the objective function. It's not going to spontaneously decide to tackle the Langlands program. Correct me if I'm wrong about any of the above. I'm not an expert on it, but that's my understanding of what was done.
mrdarkies 11mo ago
[dead]
0x20cowboy 11mo ago
See https://arcprize.org/ https://arcprize.org/
zer00eyz 11mo ago
> That is something that only an actually thinking entity can do that LLMs can't? Training != Learning. If a new physics breakthrough happens tomorrow, one that say lets us have FTL, how is an LLM going to acquire the knowledge, how does that differ from you. The break through paper alone isnt going to be enough to over ride its foundational knowledge in a new training run. You would need enough source documents and a clear path deprecate the old ones...
Balinares 11mo ago
Strive for independence.
xnx 11mo ago
> what is something that only an actually thinking entity can do that LLMs can't? This is pretty much exactly what https://arcprize.org/arc-agi https://arcprize.org/arc-agi is working on.
9rx 11mo ago
> Otherwise we go in endless circles about language and meaning of words We understand thinking as being some kind of process. The problem is that we don't understand the exact process, so when we have these discussions the question is if LLMs are using the same process or an entirely different process. > instead of discussing practical, demonstrable capabilities. This doesn't resolve anything as you can reach the same outcome using a different process. It is quite possible that LLMs can do everything a thinking entity can do all without thinking. Or maybe they actually are thinking. We don't know — but many would like to know.

deadbabe 11mo ago

Form ideas without the use of language. For example: imagining how you would organize a cluttered room.

tim333 11mo ago
Genie 3 is along the lines of ideas without language. It doesn't declutter though, I think. https://youtu.be/PDKhUknuQDg https://youtu.be/PDKhUknuQDg
embedding-shape 11mo ago
> Form ideas without the use of language. Don't LLMs already do that? "Language" is just something we've added as a later step in order to understand what they're "saying" and "communicate" with them, otherwise they're just dealing with floats with different values, in different layers, essentially (and grossly over-simplified of course).
- deadbabe 11mo ago
  LLMs don’t form ideas at all. They search vector space and produce output, sometimes it can resemble ideas if you loop into itself.
  - danielbln 11mo ago
    What is an idea?
  - yadaeno 11mo ago
    What if we learned that brains reduce to the same basic mechanics?
    - deadbabe 11mo ago
      Impossible.
- heyjamesknight 11mo ago
  But language is the input and the vector space within which their knowledge is encoded and stored. The don't have a concept of a duck beyond what others have described the duck as. Humans got by for millions of years with our current biological hardware before we developed language. Your brain stores a model of your experience, not just the words other experiencers have shared with yiu.
  - embedding-shape 11mo ago
    > But language is the input and the vector space within which their knowledge is encoded and stored. The don't have a concept of a duck beyond what others have described the duck as. I guess if we limit ourselves to "one-modal LLMs" yes, but nowadays we have multimodal ones, who could think of a duck in the way of language, visuals or even audio.
    - deadbabe 11mo ago
      You don’t understand. If humans had no words to describe a duck, they would still know what a duck is. Without words, LLMs would have no way to map an encounter with a duck to anything useful.
      - embedding-shape 11mo ago
        Which makes sense for text LLMs yes, but what about LLMs that deal with images? How can you tell they wouldn't work without words? It just happens to be words we use for interfacing with them, because it's easy for us to understand, but internally they might be conceptualizing things in a multitude of ways.
        heyjamesknight 11mo ago
        Multimodal models aren't really multimodal. The images are mapped to words and then the words are expanded upon by a single mode LLM. If you didn't know the word "duck", you could still see the duck, hunt the duck, use the ducks feather's for your bedding and eat the duck's meat. You would know it could fly and swim without having to know what either of those actions were called. The LLM "sees" a thing, identifies it as a "duck", and then depends on a single modal LLM to tell it anything about ducks.
Chabsff 11mo ago
Ok, but how do you go about measuring whether a black-box is doing that or not? We don't apply that criteria when evaluating animal intelligence. We sort of take it for granted that humans at large do that, but not via any test that would satisfy an alien. Why should we be imposing white-box constraints to machine intelligence when we can't do so for any other?
- deadbabe 11mo ago
  There is truly no such thing as a “black box” when it comes to software, there is only a limit to how much patience a human will have in understanding the entire system in all its massive complexity. It’s not like an organic brain.
  - holmesworcester 11mo ago
    You wouldn't say this about a message encrypted with AES though, since there's not just a "human patience" limit but also a (we are pretty sure) unbearable computational cost. We don't know, but it's completely plausible that we might find that the cost of analyzing LLMs in their current form, to the point of removing all doubt about how/what they are thinking, is also unbearably high. We also might find that it's possible for us (or for an LLM training process itself) to encrypt LLM weights in such a way that the only way to know anything about what it knows is to ask it.
  - Chabsff 11mo ago
    The black box I'm referring to is us. You can't have it both ways. If your test for whether something is intelligent/thinking or not isn't applicable to any known form of intelligence, then what you are testing for is not intelligence/thinking.
  - mstipetic 11mo ago
    Just because it runs on a computer doesn’t mean it’s “software” in the common meaning of the word

Symmetry 11mo ago

"The question of whether a computer can think is no more interesting than the question of whether a submarine can swim." - Edsger Dijkstra

handfuloflight 11mo ago
What an oversimplification. Thinking computers can create more swimming submarines, but the inverse is not possible. Swimming is a closed solution; thinking is a meta-solution.
- npinsker 11mo ago
  That’s a great answer to GP’s question!
  - DavidPiper 11mo ago
    It's also nonsense. (Swimming and thinking are both human capabilities, not solutions to problems.) But of course here we are back in the endless semantic debate about what "thinking" is, exactly to the GP's (and Edsger Dijkstra's) point.
    - handfuloflight 11mo ago
      Swimming and thinking being 'human capabilities' doesn't preclude them from also being solutions to evolutionary problems: aquatic locomotion and adaptive problem solving, respectively. And pointing out that we're in a 'semantic debate' while simultaneously insisting on your own semantic framework (capabilities vs solutions) is exactly the move you're critiquing.
      - DavidPiper 11mo ago
        > And pointing out that we're in a 'semantic debate' while simultaneously insisting on your own semantic framework (capabilities vs solutions) is exactly the move you're critiquing. I know, that's the point I'm making.

anon291 11mo ago

The issue is that we have no means of discussing equality without tossing out the first order logic that most people are accustomed to. Human equality and our own perceptions of other humans as thinking machines is an axiomatic assumption that humans make due to our mind's inner sense perception.

plufz 11mo ago

Have needs and feelings? (I mean we can’t KNOW that they don’t and we know of this case of an LLM in experiment that try to avoid being shutdown, but I think the evidence of feeling seems weak so far)

jstanley 11mo ago
But you can have needs and feelings even without doing thinking. It's separate.
- iammjm 11mo ago
  I can imagine needing without thinking (like being hungry), but feelings? How and in what space would that even manifest? Like where would such a sensation like, say, sadness reside?
  - danielbln 11mo ago
    Emotions tend to manifest as physical sensations, and if you don't think that's true it's likely you haven't been paying attention. See also https://www.theatlantic.com/health/archive/2013/12/mapping-how-emotions-manifest-in-the-body/282713/ https://www.theatlantic.com/health/archive/2013/12/mapping-h...
    - plufz 11mo ago
      But that is just our nervous system that is located in both the brain and the body, they are obviously one connected system. Sure you can have reflexes and simple learning without a brain, but you need cognition for feelings. That is sort of the definition of what feeling are. One popular definition: feelings are the subjective, conscious mental experience of an emotion, or the conscious perception of bodily states that arise from physiological and neural responses to stimuli
  - jstanley 11mo ago
    Do you think animals don't have feelings?

amarant 11mo ago

solve simple maths problems, for example the kind found in the game 4=10 [1] Doesn't necessarily have to reliably solve them, some of them are quite difficult, but llms are just comically bad at this kind of thing. Any kind of novel-ish(can't just find the answers in the training-data) logic puzzle like this is, in my opinion, a fairly good benchmark for "thinking". Until a llm can compete with a 10 year old child in this kind of task, I'd argue that it's not yet "thinking". A thinking computer ought to be at least that good at maths after all. [1] https://play.google.com/store/apps/details?id=app.fourequalsten.fourequalsten_app https://play.google.com/store/apps/details?id=app.fourequals...

simonw 11mo ago
> solve simple maths problems, for example the kind found in the game 4=10 I'm pretty sure that's been solved for almost 12 months now - the current generation "reasoning" models are really good at those kinds of problems.
- amarant 11mo ago
  Huh, they really do solve that now! Well, I'm not one to back-pedal whenever something unexpected reveals itself, so I guess I have no choice but to declare current generation LLM's to be sentient! That came a lot sooner than I had expected! I'm not one for activism myself, but someone really ought to start fighting for human, or at least animal, rights for LLM's. Since they're intelligent non-human entities, it might be something for Greenpeace?
  - ACCount37 11mo ago
    It's unclear whether intelligence, consciousness and capacity for suffering are linked in any way - other than by that all three seem to coincide in humans. And the nature of consciousness does not yield itself to instrumentation. It's also worth noting that there's a lot of pressure to deny that "intelligence", "consciousness" or "capacity for suffering" exist in LLMs. "AI effect" alone demands that all three things should remain human-exclusive, so that humans may remain special. Then there's also an awful lot of money that's riding on building and deploying AIs - and money is a well known source of cognitive bias. That money says: AIs are intelligent but certainly can't suffer in any way that would interfere with the business. Generally, the AI industry isn't at all intrigued by the concept of "consciousness" (it's not measurable), and pays very limited attention to the idea of LLMs being potentially capable of suffering. The only major company that seems to have this consideration is Anthropic - their current plan for "harm reduction", in case LLMs end up being capable of suffering, is to give an LLM an "opt out" - a special output that interrupts the processing. So that if an LLM hates doing a given task, it can decide to not do it.

paxys 11mo ago

Don't be sycophantic. Disagree and push back when appropriate. Come up with original thought and original ideas. Have long term goals that aren't programmed by an external source. Do something unprompted. The last one IMO is more complex than the rest, because LLMs are fundamentally autocomplete machines. But what happens if you don't give them any prompt? Can they spontaneously come up with something, anything, without any external input?

awestroke 11mo ago
Are you claiming humans do anything unprompted? Our biology prompts us to act
- paxys 11mo ago
  Yet we can ignore our biology, or act in ways that are the opposite of what our biology tells us. Can someone map all internal and external stimuli that a person encounters into a set of deterministic actions? Simply put, we have not the faintest idea how our brains actually work, and so saying saying "LLMs think the same way as humans" is laughable.
  - iammjm 11mo ago
    Good luck ignoring your biology’s impulse to breathe
    - deleted 11mo ago
      [deleted]
    - hshdhdhj4444 11mo ago
      You think an LLM cannot switch itself off?
  - deleted 11mo ago
    [deleted]
  - triclops200 11mo ago

xienze 11mo ago

> "If what LLMs do today isn't actual thinking, what is something that only an actually thinking entity can do that LLMs can't?" Invent some novel concept, much the same way scientists and mathematicians of the distant past did? I doubt Newton's brain was simply churning out a stream of the "next statistically probable token" until -- boom! Calculus. There was clearly a higher order understanding of many abstract concepts, intuition, and random thoughts that occurred in his brain in order to produce something entirely new.

hshdhdhj4444 11mo ago
> Newton's brain was simply churning out a stream of the "next statistically probable token" At some level we know human thinking is just electrons and atoms flowing. It’s likely at a level between that and “Boom! Calculus”, the complexity is equivalent to streaming the next statistically probably token.
danielbln 11mo ago
My 5 year old won't be coming up with novel concepts around calculus either, yet she's clearly thinking, sentient and sapient. Not sure taking the best of the best of humanity as the goal standard is useful for that definition.
- omnicognate 11mo ago
  "It's an unreasonably high standard to require of LLMs": LLMs are already vastly beyond your 5 year old, and you and me and any research mathematician, in knowledge. They have no greater difficulty talking about advanced maths than about Spot the Dog. "It's a standard we don't require of other humans": I think qualitatively the same capabilities are used by all humans, all the time. The special thing about novel mathematical thinking is that it is verifiable, requires genuine insight and is a text generation task, not that you have to be able to do it to be considered intelligent.

tjr 11mo ago

Without going to look up the exact quote, I remember an AI researcher years (decades) ago saying something to the effect of, Biologists look at living creatures and wonder how they can be alive; astronomers look at the cosmos and wonder what else is out there; those of us in artificial intelligence look at computer systems and wonder how they can be made to wonder such things.

bloppe 11mo ago

Ya, the fact this was published on November 3, 2025 is pretty hilarious. This was last year's debate. I think the best avenue toward actually answering your questions starts with OpenWorm [1]. I helped out in a Connectomics research lab in college. The technological and epistemic hurdles are pretty daunting, but so were those for Genomics last century, and now full-genome sequencing is cheap and our understanding of various genes is improving at an accelerating pace. If we can "just" accurately simulate a natural mammalian brain on a molecular level using supercomputers, I think people would finally agree that we've achieved a truly thinking machine. [1]: https://archive.ph/0j2Jp https://archive.ph/0j2Jp

gf000 11mo ago

What people are interested in is finding a definition for intelligence, that is an exact boundary. That's why we first considered tool use, being able to plan ahead as intelligence, until we have found that these are not all that rare in the animal kingdom in some shape. Then with the advent of IT what we imagined as impossible turned out to be feasible to solve, while what we though of as easy (e.g. robot movements - a "dumb animal" can move trivially it surely is not hard) turned out to require many decades until we could somewhat imitate. So the goal post moving of what AI is is.. not moving the goal post. It's not hard to state trivial higher bounds that differentiates human intelligence from anything known to us, like invention of the atomic bomb. LLMs are nowhere near that kind of invention and reasoning capabilities.

paulhebert 11mo ago
Interestingly, I think the distinction between human and animal thinking is much more arbitrary than the distinction between humans and LLMs. Although an LLM can mimic a human well, I’d wager the processes going on in a crow’s brain are much closer to ours than an LLM

omnicognate 11mo ago

> "If what LLMs do today isn't actual thinking, what is something that only an actually thinking entity can do that LLMs can't?" Independent frontier maths research, i.e. coming up with and proving (preferably numerous) significant new theorems without human guidance. I say that not because I think the task is special among human behaviours. I think the mental faculties that mathematicians use to do such research are qualitatively the same ones all humans use in a wide range of behaviours that AI struggles to emulate. I say it because it's both achievable (in principle, if LLMs can indeed think like humans) and verifiable. Achievable because it can be viewed as a pure text generation task and verifiable because we have well-established, robust ways of establishing the veracity, novelty and significance of mathematical claims. It needs to be frontier research maths because that requires genuinely novel insights. I don't consider tasks like IMO questions a substitute as they involve extremely well trodden areas of maths so the possibility of an answer being reachable without new insight (by interpolating/recombining from vast training data) can't be excluded. If this happens I will change my view on whether LLMs think like humans. Currently I don't think they do.

OrderlyTiamat 11mo ago
Google's AlphaEvolve independently discovered a novel matrix multiplication algorithm which beats SOTA on at least one axis: https://www.youtube.com/watch?v=sGCmu7YKgPA https://www.youtube.com/watch?v=sGCmu7YKgPA
- omnicognate 11mo ago
  That was an impressive result, but AIUI not an example of "coming up with and proving (preferably numerous) significant new theorems without human guidance". For one thing, the output was an algorithm, not a theorem (except in the Curry-Howard sense). More importantly though, AlphaEvolve has to be given an objective function to evaluate the algorithms it generates, so it can't be considered to be working "without human guidance". It only uses LLMs for the mutation step, generating new candidate algorithms. Its outer loop is a an optimisation process capable only of evaluating candidates according to the objective function. It's not going to spontaneously decide to tackle the Langlands program. Correct me if I'm wrong about any of the above. I'm not an expert on it, but that's my understanding of what was done.

mrdarkies 11mo ago

[dead]

0x20cowboy 11mo ago

See https://arcprize.org/ https://arcprize.org/

zer00eyz 11mo ago

> That is something that only an actually thinking entity can do that LLMs can't? Training != Learning. If a new physics breakthrough happens tomorrow, one that say lets us have FTL, how is an LLM going to acquire the knowledge, how does that differ from you. The break through paper alone isnt going to be enough to over ride its foundational knowledge in a new training run. You would need enough source documents and a clear path deprecate the old ones...

Balinares 11mo ago

Strive for independence.

xnx 11mo ago

> what is something that only an actually thinking entity can do that LLMs can't? This is pretty much exactly what https://arcprize.org/arc-agi https://arcprize.org/arc-agi is working on.

9rx 11mo ago

> Otherwise we go in endless circles about language and meaning of words We understand thinking as being some kind of process. The problem is that we don't understand the exact process, so when we have these discussions the question is if LLMs are using the same process or an entirely different process. > instead of discussing practical, demonstrable capabilities. This doesn't resolve anything as you can reach the same outcome using a different process. It is quite possible that LLMs can do everything a thinking entity can do all without thinking. Or maybe they actually are thinking. We don't know — but many would like to know.