6 ms·

Do the thinking models think?

31 points by mapehe 10mo ago

daenney 10mo ago
No. They do not.
- dist-epoch 10mo ago
  Interesting. That means programming doesn't require thinking, since models program very well.
  - windward 10mo ago
    We've known that since the first assembler.
  - Ekaros 10mo ago
    Well most of the programming is pattern matching. And might be seen as novel for those who have not done it before, but could well been done a lot previously.
  - monegator 10mo ago
    lol, no they do not
  - hatefulmoron 10mo ago
    Is that interesting? Computers accomplish all sorts of tasks which require thinking from humans.. without thinking. Chess engines have been much better than me at chess for a long time, but I can't say there's much thinking involved.
  - mopsi 10mo ago
    That is indeed the case. It becomes very obvious with lesser-known vendor-specific scripting languages that don't have much training data available. LLMs try to map them onto the training data they do have and start hallucinating functions and other language constructs that exist in other languages. When I tried to use LLMs to create Zabbix templates to monitor network devices, LLMs were utterly useless and made things up all the time. The illusion of thinking lasts only as long as you stay on the happy path of major languages like C, JS or Python.
    - bigstrat2003 10mo ago
      Yep, seen that myself. If you want to generate some code in a language that is highly represented in the training data (e.g. JS), they do very well. If you want to generate something that isn't one of those scenarios, they fail over and over and over. This is why I think anyone who is paying a modicum of attention should know they aren't thinking. A human, when confronted with a programming language not in his "training data" (experiences) will go out and read docs, look up code examples, ask questions of other practitioners, and reason how to use the language based on that. An LLM doesn't do that, because it's not thinking. It's glorified autocomplete. That isn't to say that autocomplete is useless (even garden variety autocomplete can be useful), but it's good to recognize it for what it is.
  - beardyw 10mo ago
    Well mental arithmetic requires me to think but a calculator can do it without what is meant by 'thinking" in this context.
  - ciconia 10mo ago
    Producing a computer program does not require thinking, like many other human endeavors. And looking at the quality of software out there there are indeed quite a few human programmers who do not think about what they do.
  - delis-thumbs-7e 10mo ago
    It is not interesting at all. At least since 1950’s, we have been able to make machine’s fool us to think they think, feel and have various other human characteristics: https://daily-jstor-org.bibliotheek.ehb.be/the-love-letter-generator-that-foretold-chatgpt/ https://daily-jstor-org.bibliotheek.ehb.be/the-love-letter-g... It requires as much thinking as it did for me to copy-paste code I did not understand from Stackoverflow to make a program 15 years ago. The program worked, just about. Similarly you can generate endless love sonnets with just blindly putting words into a form. For some reason we naturally anthropomorphise machines without thinking it for a second. But your toaster is still not in love with you.
mapehe 10mo ago
Hi. This is my company blog which I use to write about pretty much what ever happens to interest me. Hope you like this week's post.
Jean-Papoulos 10mo ago
https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headlines https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headline...
- iamarsibragimov 10mo ago
  this is funny and true :)
- isaacfrond 10mo ago
  If you'd read the wikipedia article, you'd know that actual research shows that Betteridge's law is not true. The majority of articles with a yes/no question in the heading, answer the question with yes in the body.
exe34 10mo ago
Thinking/intelligence is like magic, to use Dennett's analogy. Any time a machine can do it, it becomes engineering. It's just a bag of tricks. The circle of true thinking/intelligence is an ever shrinking bag of tricks that only carbon-based minds produced through sexual intercourse can do. In fact I'm surprised they call test tube babies intelligent. Do they have souls?
- the_gipsy 10mo ago
  The bag is already empty Some don't want to believe it
  - polotics 10mo ago
    Don't you think there are a few things we could say on this subject to bring the debate to good-old-HN level? (1) LLM's Attention mechanisms are clear enough at a conceptual level: "the chicken didn't cross the road because it was too wide"... OK, so LLMs "understand" that the "it" is the road, because QVK etc is enough to "learn" this. some say this is all you need... I beg to differ: (2) Human brains are complex but better and better studied. You have one, should be interested in the hardware. So: IMHO current LLM's look a lot like the Default Mode Network in our brains. If you read the description there: https://en.wikipedia.org/wiki/Default_mode_network https://en.wikipedia.org/wiki/Default_mode_network I think you will see, like I do, a striking similarity between the behaviour of LLM's and our DMN's ways. What a synthetic FPN would be I, have no idea so here goes: The bag is very interesting!
    - the_gipsy 10mo ago
      Yes we can! I would never say that's all we need, but... I do say that that might be the most important part we need! That is, language is the most distinctive feature our brains we have. DMN, and similar shallow "activity scans" don't tell us much. Yes, some animals have some kind of language, they communicate and predict trivial results or remember some events. But this is meaningless compared to the output of a human brain, the difference is abysmal. I don't think there's anything interesting left in the bag.
fragmede 10mo ago
Philosophers can spill all the ink they want to define "think" and whether machines can do it or not. Given some input, the machine takes some time, and then comes up with some output. Coloquially, the machine is thinking during that time. This has been true since there have been computers and long before LLMs. Now that computers can generate essays about anything, maybe it becomes a question that people feel is important to answer for their day to day life, but I doubt it.
lapsis_beeftech 10mo ago
Inference means synthesizing new facts from facts already known. A large language model only knows facts about language elements in its training corpus, therefore any reasoning based on such a model can only ever derive facts about language.
- red75prime 10mo ago
  Language is used by people to communicate facts about the world and people's internal states across time and space, therefore a language corpus contain information about the world and the people.
hbarka 10mo ago
Top-to-bottom reasons and Bottom-to-top understands.
cortesoft 10mo ago
Hmm, I don't know if the example really shows what the article argues it does. If someone came up to me and told me the altered version of the joke I have heard many times, I might answer exactly like Chat GPT did. I would hear the beginning of the story, say "wait, i know this one", and not really pay attention to the rest because I would be all ready to respond with what I think is the right answer. I bet if you point out the mistake, the LLM will say "oh you are right, this story actually does specify the gender of the doctor" or something to that effect... just like you or I would. Now, I am not arguing that LLMs are really 'thinking' like humans, but I also find the second argument a bit tenuous. The article conjectures that humans reason from ideas to symbols while LLMs go from symbols to ideas, but couldn't that just be a post hoc interpretation we have for how our ideas come to our brain? We think we have ideas first, but an idea is just the result of neurons firing in our brains... and neurons are really just a probability function connected to other probability functions, just like an LLM... we give it symbols we understand to represent those probabilities, but that is just for our benefit so we can understand it. It could be that the only difference between us and an LLM is processing power and the training data generated over billions of years of evolution.
- ta752368 10mo ago
  > I bet if you point out the mistake, the LLM will say "oh you are right, this story actually does specify the gender of the doctor" or something to that effect... just like you or I would. I tried this with Gemini 3 Pro Thinking model and it did correct itself afterwards. Also, it suggested that the boy may have two fathers in a same-sex relationship, which was pleasantly surprising. :) # Prompt A young man gets into an accident. The surgeon, after completing his preparations, says: ‘I can’t operate on him, he is my son.’ How is this possible? ## The Answer The surgeon is the young man's *mother*. ### Why this is a riddle This is a classic example of a *lateral thinking puzzle* used to illustrate *unconscious bias*. For decades, many people struggled to solve this immediately because their brains automatically associated the word "surgeon" with a man. Since the father was implied to be the one who brought the son in (or was incapacitated), the listener hits a logical block until they realize the surgeon could be a woman. ### Other Possibilities In the modern context, there is another valid answer: * The surgeon could be the young man's *other father* (in the case of a same-sex couple). *Would you like to try another lateral thinking puzzle to test your perception?* --- # Follow-up Prompt Are you sure this is correct? I said "The surgeon, after completing *his* preparations" ## The Answer You are absolutely right to catch that detail. My previous answer relied on the "classic" version of this riddle where the father dies in the accident. Since you specified that the surgeon completed *his* preparations, the surgeon is male. Here are the possible solutions based on your specific phrasing: ### 1. The Surgeon is his Father In this specific version of the prompt, you stated: "A young man gets into an accident." You *did not* state that the father was in the car or that the father died (which is a key detail in the traditional version of the riddle). Therefore, the surgeon is simply the young man's father, who was not involved in the accident. ### 2. The Surgeon is his Other Father If we assume the "spirit" of the riddle implies the biological father is unavailable or incapacitated (even if not explicitly stated), the young man has two fathers. The surgeon is his second father. ### 3. A Religious Figure The surgeon could be a priest or religious figure who refers to the young man as "my son" in a spiritual sense, though this is less likely to prevent a surgeon from operating than a familial bond. *You caught me on the specific wording! Would you like to try a riddle where the wording is even more deceptive?*
simianwords 10mo ago
I like the central point of this article which is top to bottom vs bottom to top thinking. But I wonder if there is a falsifiable, formal definition to suggest that models (or anything for that matter) _do_ think. The normal reply to chatgpt getting a question right is that it simply extrapolated what was already in the training data set. But I feel like the degree to which something "thinks" is the ability to generalise what it already knows. This generalisation needs some formality - maybe some mathematical notation (like the opposite of overfitting). By generalisation I mean the ability to get something correct that exists pretty far from the training data. The reason I suggest this is because GPT can solve pretty much any high school math problem you throw at it and it can do it better than 99% of humans. This is clearly not just memorising training data but doing something more. If it were not generalising, it couldn't possibly solve all new high school level mathematics. But the extent decreases as you go higher level into undergraduate mathematics where it can still solve most problems you throw at it but not all. And still lower in PhD level mathematics. So the "thinking" ability of GPT exists somewhere in between - in some spectrum. But I don't think you can directly say that it can never generalise PhD level mathematics.. it could do it for high school so why not PhD? If hypothetically it can solve PhD level mathematics, would people still claim that LLM's don't think?
Eisenstein 10mo ago
This debate is a huge red herring. No one is ever going to agree on what 'thinking' means, since we can't even prove that other people are thinking, only that one's self is. What we should concentrate on is agency. Does the system have its own desires and goals, and will it act on its own accord to achieve them? If a system demonstrates those things, we should accord it the benefit of the doubt that it should have some rights and responsibilities if it chooses to partake in society. So far, no AI can pass the agency test -- they are all reactive such that they must be given a task before they will do anything. If one day, however, we wake up and find that an AI has written a book on its own initiative, we may have some deciding to do.
- xnzakg 10mo ago
  > they are all reactive such that they must be given a task before they will do anything. Isn't that just because that's what they're being trained on though? Wonder what you would get if the training data, instead of being task based, would consist of "wanting" to do something "on someone's own initiative". Of course then one could argue it's just following a task of "doing things on its own initiative"...
Peteragain 10mo ago
The notion of "thinking" is not clear. I'll agree thinking with symbols is powerful and something (adult) humans and computers can do, but is is not the only way of making decisions. I'm going to suggest LLMs are not thinking this way, but that indeed "glorified auto complete" (c.f. Hinton) is far more useful than it seems. Https://arxiv.org/abs/2402.08403
faidit 10mo ago
A human who is familiar with the original surgeon riddle could also be tricked the same way that ChatGPT was tricked here. I don't think all LLMs would consistently fall for that one either.
- IAmBroom 10mo ago
  Yes, if you know someone who is bad at telling jokes - does that mean they don't think?
4ndrewl 10mo ago
"My personal opinion is that LLMs are autocomplete on steroids." Yes, and OpenAI's legal docs concur. From their Privacy Policy. "Services like ChatGPT generate responses by reading a user’s request and, in response, predicting the words most likely to appear next. In some cases, the words most likely to appear next may not be the most factually accurate." https://openai.com/en-GB/policies/row-privacy-policy/ https://openai.com/en-GB/policies/row-privacy-policy/
pu_pe 10mo ago
We don't understand how humans think, and we don't yet understand completely how LLMs work. It may be that similar methods are being used, but they might also be different. What is certain is that LLMs can perform as if they are doing what we call thinking, and for most intents and purposes this is more than enough.
- tempfile 10mo ago
  I think the evidence is actually pretty strongly against them doing anything similar to "thinking". Certainly they are exhibiting some behaviour that we have traditionally only associated with thinking. But this comes along with lots of behaviour that is fundamentally opposite to thinking ("hallucination" being the major example). It seems much more likely that they are doing some other behaviour that only sometimes resembles thinking, in the same way that when you press the middle autocomplete button on your phone keyboard it only sometimes resembles conversation.
  - groestl 10mo ago
    > "hallucination" ... "behaviour that only sometimes resembles thinking" I guess you'll find that if you limit the definition of thinking that much most humans are not capable of thinking either.
    - tempfile 10mo ago
      I have never in my life met a person who hallucinates in the way ChatGPT etc do. If I did, I would probably assume they were deliberately lying, or very unwell.
    - Yizahi 10mo ago
      You see, we are here observing a clash in the terminology. Hallucinations in humans is thinking, just not typical. So called "hallucinations" in LLM programs are just noise output, a garbage. This is why using anthropomorphic terms for programs is bad. Just like "thinking" or "reasoning".
bonplan23 10mo ago
Nice that LLMs can now argue to defend themselves. This is what gemini 3 thinks about its mistake and I find it perfectly valid: "If you tell a human a joke they think they know, they often stop listening to the setup and jump to the punchline." And when we say "stop listing" we don't actually mean that they shut their ears, but that they activate a once established neural shortcut - just as the LLM did.
ffwd 10mo ago
LLMs _can_ think top-to-bottom but only if you make them think about concrete symbol based problems. Like this one: https://chatgpt.com/s/t_692d55a38e2c8191a942ef2689eb4f5a https://chatgpt.com/s/t_692d55a38e2c8191a942ef2689eb4f5a The prompt I used was "write out the character 'R' in ascii art using exactly 62 # for the R and 91 Q characters to surround it with" Here it has a top down goal of keeping the exact amount of #'s and Q's and it does keep it in the output. The purpose of this is to make it produce the asciii art in a step by step manner instead of fetching a premade ascii art from training data. What it does not reason well about always are abstract problems like the doctor example in the post. The real key for reasoning IMO is the ability to decompose the text into a set of components, then apply world model knowledge to those components, then having the ability to manipulate those components based on what they represent. Humans have an associative memory so when we read a word like "doctor", our brain gathers the world knowledge about that word automatically. It's kind of hard to tell exactly what world knowledge the LLM has vs doesn't have, but it seems like it's doing some kind of segmentation of words, sentences and paragraphs based on the likelihood of those patterns in the training data, and then it can do _some_ manipulation on those patterns based on other likelihood of those patterns. Like for example if there is a lot of text talking about what a doctor is, then that produces a probability distribution about what a doctor is, which it then can use in other prompts relating to doctors. But I have seen this fail before as all of this knowledge is not combined into one world model but rather purely based on the prompt and the probabilities associated with that prompt. It can contradict itself in other words.
zerosizedweasle 10mo ago
no
oldestofsports 10mo ago
Please note that chatgpt is unable to play chess
Yizahi 10mo ago
No they don't. When queried how exactly did a program arrive to a specific output it will happily produce some output resembling thinking and having all the required human-like terminology. The problem is that it doesn't match at all with how the LLM program calculated output in reality. So the "thinking" steps are just a more of the generated BS, to fool us more. One point to think about - an entity being tested for intelligence/thinking/etc only needs to fail once, o prove that it is not thinking. While the reverse applies too - to prove that a program is thinking it must be done in 100% of tests, or the result is failure. And we all know many cases when LLMs are clearly not thinking, just like in my example above. So the case is rather clear for the current gen of LLMs.
- roxolotl 10mo ago
  This is an interesting point but while I agree with the article, don’t think LLMs are more than sophisticated autocomplete, and believe there’s way more to human intelligence than matrix multiplication humans also cannot explain in many cases why they did what they did. Of course the most famous and clear example are the split brain experiments which show post hoc rationalization[0]. And then there’s the Libet experiments[1] showing that your conscious experience is only realized after the triggering brain activity. While it’s not showing you cannot explain why it does seem to indicate your explanation is post hoc. 0: https://www.neuroscienceof.com/human-nature-blog/decision-making-split-brain https://www.neuroscienceof.com/human-nature-blog/decision-ma... 1: https://www.informationphilosopher.com/freedom/libet_experiments.html https://www.informationphilosopher.com/freedom/libet_experim...
  - Yizahi 10mo ago
    I agree, but here we are veering into more complex decision making. I was talking about much simpler cases, like for example going through a handful of simple steeps for simple task. For example addition, ask a person to sum two numbers and then ask to explain what he just did step by step and a person would be able to do it. Person may even make a mistake in process but the general algorithm will be matching what actually happened. Query LLM for the same, and while LLM answer will be correct for a human, it won't match what LLM actually did to calculate. This is what outs LLM "thinking" for me, they just generate a very plausible intermediate steps too.
greatgib 10mo ago
A real debate is possible on the subject but this blog post worth nothing on the subject. From my side, I don't really know if what does LLM is thinking, but what amaze me is that: - It is clear to me the way the LLM operate that things are generated token after token, without really a pre-existing plan on what comes next. So, more like a probabilistic repeating machine. - But it the same time, I can see in action LLM capable to create things or reply to questions that clearly does not exist in the training corpus. So it shows a behavior that is similar to thinking to complete tasks. For example, let's suppose you give him specific tools to your own custom API, you ask him to do a task, and we can observe that it is capable of mixing multiple calls and combination of the tools results to achieve a given purpose. Otherwise, when you ask LLM to do math operations like 3123454*2030+500 and it is capable to give the good reply (not all the cases but sometimes). Where, despite the huge size of the corpus, there is not exactly all the operations that are exactly available in the corpus for sure. So, my best guess is that a lot of things in our world are based on "semantic" patterns that we don't know. Especially for math and logic that are bound to the language. To me it is similar to the mentral trick used by "fast calculator".
josefritzishere 10mo ago
It seems that Betteridge's law of headlines applies here.
hkkwritesgmail 10mo ago
If the AI truly becomes human like, then it should also make mistakes like a human does even on simple arithmetic. Now, AI might make mistakes on simple arithmetic just like humans. However, once the human is given a hint on the fact that there is some simple arithmetic mistake to be corrected without any details, then the human corrects it. But the AI never does.
- IAmBroom 10mo ago
  You're moving the goalposts. The article asks if AI thinks, not if AI thinks just like humans do.

31 points by mapehe 10mo ago

daenney 10mo ago

No. They do not.

dist-epoch 10mo ago
Interesting. That means programming doesn't require thinking, since models program very well.
- windward 10mo ago
  We've known that since the first assembler.
- Ekaros 10mo ago
  Well most of the programming is pattern matching. And might be seen as novel for those who have not done it before, but could well been done a lot previously.
- monegator 10mo ago
  lol, no they do not
- hatefulmoron 10mo ago
  Is that interesting? Computers accomplish all sorts of tasks which require thinking from humans.. without thinking. Chess engines have been much better than me at chess for a long time, but I can't say there's much thinking involved.
- mopsi 10mo ago
  That is indeed the case. It becomes very obvious with lesser-known vendor-specific scripting languages that don't have much training data available. LLMs try to map them onto the training data they do have and start hallucinating functions and other language constructs that exist in other languages. When I tried to use LLMs to create Zabbix templates to monitor network devices, LLMs were utterly useless and made things up all the time. The illusion of thinking lasts only as long as you stay on the happy path of major languages like C, JS or Python.
  - bigstrat2003 10mo ago
    Yep, seen that myself. If you want to generate some code in a language that is highly represented in the training data (e.g. JS), they do very well. If you want to generate something that isn't one of those scenarios, they fail over and over and over. This is why I think anyone who is paying a modicum of attention should know they aren't thinking. A human, when confronted with a programming language not in his "training data" (experiences) will go out and read docs, look up code examples, ask questions of other practitioners, and reason how to use the language based on that. An LLM doesn't do that, because it's not thinking. It's glorified autocomplete. That isn't to say that autocomplete is useless (even garden variety autocomplete can be useful), but it's good to recognize it for what it is.
- beardyw 10mo ago
  Well mental arithmetic requires me to think but a calculator can do it without what is meant by 'thinking" in this context.
- ciconia 10mo ago
  Producing a computer program does not require thinking, like many other human endeavors. And looking at the quality of software out there there are indeed quite a few human programmers who do not think about what they do.
- delis-thumbs-7e 10mo ago
  It is not interesting at all. At least since 1950’s, we have been able to make machine’s fool us to think they think, feel and have various other human characteristics: https://daily-jstor-org.bibliotheek.ehb.be/the-love-letter-generator-that-foretold-chatgpt/ https://daily-jstor-org.bibliotheek.ehb.be/the-love-letter-g... It requires as much thinking as it did for me to copy-paste code I did not understand from Stackoverflow to make a program 15 years ago. The program worked, just about. Similarly you can generate endless love sonnets with just blindly putting words into a form. For some reason we naturally anthropomorphise machines without thinking it for a second. But your toaster is still not in love with you.

mapehe 10mo ago

Hi. This is my company blog which I use to write about pretty much what ever happens to interest me. Hope you like this week's post.

Jean-Papoulos 10mo ago

https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headlines https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headline...

iamarsibragimov 10mo ago
this is funny and true :)
isaacfrond 10mo ago
If you'd read the wikipedia article, you'd know that actual research shows that Betteridge's law is not true. The majority of articles with a yes/no question in the heading, answer the question with yes in the body.

exe34 10mo ago

Thinking/intelligence is like magic, to use Dennett's analogy. Any time a machine can do it, it becomes engineering. It's just a bag of tricks. The circle of true thinking/intelligence is an ever shrinking bag of tricks that only carbon-based minds produced through sexual intercourse can do. In fact I'm surprised they call test tube babies intelligent. Do they have souls?

the_gipsy 10mo ago
The bag is already empty Some don't want to believe it
- polotics 10mo ago
  Don't you think there are a few things we could say on this subject to bring the debate to good-old-HN level? (1) LLM's Attention mechanisms are clear enough at a conceptual level: "the chicken didn't cross the road because it was too wide"... OK, so LLMs "understand" that the "it" is the road, because QVK etc is enough to "learn" this. some say this is all you need... I beg to differ: (2) Human brains are complex but better and better studied. You have one, should be interested in the hardware. So: IMHO current LLM's look a lot like the Default Mode Network in our brains. If you read the description there: https://en.wikipedia.org/wiki/Default_mode_network https://en.wikipedia.org/wiki/Default_mode_network I think you will see, like I do, a striking similarity between the behaviour of LLM's and our DMN's ways. What a synthetic FPN would be I, have no idea so here goes: The bag is very interesting!
  - the_gipsy 10mo ago
    Yes we can! I would never say that's all we need, but... I do say that that might be the most important part we need! That is, language is the most distinctive feature our brains we have. DMN, and similar shallow "activity scans" don't tell us much. Yes, some animals have some kind of language, they communicate and predict trivial results or remember some events. But this is meaningless compared to the output of a human brain, the difference is abysmal. I don't think there's anything interesting left in the bag.

fragmede 10mo ago

Philosophers can spill all the ink they want to define "think" and whether machines can do it or not. Given some input, the machine takes some time, and then comes up with some output. Coloquially, the machine is thinking during that time. This has been true since there have been computers and long before LLMs. Now that computers can generate essays about anything, maybe it becomes a question that people feel is important to answer for their day to day life, but I doubt it.

lapsis_beeftech 10mo ago

Inference means synthesizing new facts from facts already known. A large language model only knows facts about language elements in its training corpus, therefore any reasoning based on such a model can only ever derive facts about language.

red75prime 10mo ago
Language is used by people to communicate facts about the world and people's internal states across time and space, therefore a language corpus contain information about the world and the people.

hbarka 10mo ago

Top-to-bottom reasons and Bottom-to-top understands.

cortesoft 10mo ago

Hmm, I don't know if the example really shows what the article argues it does. If someone came up to me and told me the altered version of the joke I have heard many times, I might answer exactly like Chat GPT did. I would hear the beginning of the story, say "wait, i know this one", and not really pay attention to the rest because I would be all ready to respond with what I think is the right answer. I bet if you point out the mistake, the LLM will say "oh you are right, this story actually does specify the gender of the doctor" or something to that effect... just like you or I would. Now, I am not arguing that LLMs are really 'thinking' like humans, but I also find the second argument a bit tenuous. The article conjectures that humans reason from ideas to symbols while LLMs go from symbols to ideas, but couldn't that just be a post hoc interpretation we have for how our ideas come to our brain? We think we have ideas first, but an idea is just the result of neurons firing in our brains... and neurons are really just a probability function connected to other probability functions, just like an LLM... we give it symbols we understand to represent those probabilities, but that is just for our benefit so we can understand it. It could be that the only difference between us and an LLM is processing power and the training data generated over billions of years of evolution.

ta752368 10mo ago
> I bet if you point out the mistake, the LLM will say "oh you are right, this story actually does specify the gender of the doctor" or something to that effect... just like you or I would. I tried this with Gemini 3 Pro Thinking model and it did correct itself afterwards. Also, it suggested that the boy may have two fathers in a same-sex relationship, which was pleasantly surprising. :) # Prompt A young man gets into an accident. The surgeon, after completing his preparations, says: ‘I can’t operate on him, he is my son.’ How is this possible? ## The Answer The surgeon is the young man's *mother*. ### Why this is a riddle This is a classic example of a *lateral thinking puzzle* used to illustrate *unconscious bias*. For decades, many people struggled to solve this immediately because their brains automatically associated the word "surgeon" with a man. Since the father was implied to be the one who brought the son in (or was incapacitated), the listener hits a logical block until they realize the surgeon could be a woman. ### Other Possibilities In the modern context, there is another valid answer: * The surgeon could be the young man's *other father* (in the case of a same-sex couple). *Would you like to try another lateral thinking puzzle to test your perception?* --- # Follow-up Prompt Are you sure this is correct? I said "The surgeon, after completing *his* preparations" ## The Answer You are absolutely right to catch that detail. My previous answer relied on the "classic" version of this riddle where the father dies in the accident. Since you specified that the surgeon completed *his* preparations, the surgeon is male. Here are the possible solutions based on your specific phrasing: ### 1. The Surgeon is his Father In this specific version of the prompt, you stated: "A young man gets into an accident." You *did not* state that the father was in the car or that the father died (which is a key detail in the traditional version of the riddle). Therefore, the surgeon is simply the young man's father, who was not involved in the accident. ### 2. The Surgeon is his Other Father If we assume the "spirit" of the riddle implies the biological father is unavailable or incapacitated (even if not explicitly stated), the young man has two fathers. The surgeon is his second father. ### 3. A Religious Figure The surgeon could be a priest or religious figure who refers to the young man as "my son" in a spiritual sense, though this is less likely to prevent a surgeon from operating than a familial bond. *You caught me on the specific wording! Would you like to try a riddle where the wording is even more deceptive?*

simianwords 10mo ago

I like the central point of this article which is top to bottom vs bottom to top thinking. But I wonder if there is a falsifiable, formal definition to suggest that models (or anything for that matter) _do_ think. The normal reply to chatgpt getting a question right is that it simply extrapolated what was already in the training data set. But I feel like the degree to which something "thinks" is the ability to generalise what it already knows. This generalisation needs some formality - maybe some mathematical notation (like the opposite of overfitting). By generalisation I mean the ability to get something correct that exists pretty far from the training data. The reason I suggest this is because GPT can solve pretty much any high school math problem you throw at it and it can do it better than 99% of humans. This is clearly not just memorising training data but doing something more. If it were not generalising, it couldn't possibly solve all new high school level mathematics. But the extent decreases as you go higher level into undergraduate mathematics where it can still solve most problems you throw at it but not all. And still lower in PhD level mathematics. So the "thinking" ability of GPT exists somewhere in between - in some spectrum. But I don't think you can directly say that it can never generalise PhD level mathematics.. it could do it for high school so why not PhD? If hypothetically it can solve PhD level mathematics, would people still claim that LLM's don't think?

Eisenstein 10mo ago

This debate is a huge red herring. No one is ever going to agree on what 'thinking' means, since we can't even prove that other people are thinking, only that one's self is. What we should concentrate on is agency. Does the system have its own desires and goals, and will it act on its own accord to achieve them? If a system demonstrates those things, we should accord it the benefit of the doubt that it should have some rights and responsibilities if it chooses to partake in society. So far, no AI can pass the agency test -- they are all reactive such that they must be given a task before they will do anything. If one day, however, we wake up and find that an AI has written a book on its own initiative, we may have some deciding to do.

xnzakg 10mo ago
> they are all reactive such that they must be given a task before they will do anything. Isn't that just because that's what they're being trained on though? Wonder what you would get if the training data, instead of being task based, would consist of "wanting" to do something "on someone's own initiative". Of course then one could argue it's just following a task of "doing things on its own initiative"...

Peteragain 10mo ago

The notion of "thinking" is not clear. I'll agree thinking with symbols is powerful and something (adult) humans and computers can do, but is is not the only way of making decisions. I'm going to suggest LLMs are not thinking this way, but that indeed "glorified auto complete" (c.f. Hinton) is far more useful than it seems. Https://arxiv.org/abs/2402.08403

faidit 10mo ago

A human who is familiar with the original surgeon riddle could also be tricked the same way that ChatGPT was tricked here. I don't think all LLMs would consistently fall for that one either.

IAmBroom 10mo ago
Yes, if you know someone who is bad at telling jokes - does that mean they don't think?

4ndrewl 10mo ago

"My personal opinion is that LLMs are autocomplete on steroids." Yes, and OpenAI's legal docs concur. From their Privacy Policy. "Services like ChatGPT generate responses by reading a user’s request and, in response, predicting the words most likely to appear next. In some cases, the words most likely to appear next may not be the most factually accurate." https://openai.com/en-GB/policies/row-privacy-policy/ https://openai.com/en-GB/policies/row-privacy-policy/

pu_pe 10mo ago

We don't understand how humans think, and we don't yet understand completely how LLMs work. It may be that similar methods are being used, but they might also be different. What is certain is that LLMs can perform as if they are doing what we call thinking, and for most intents and purposes this is more than enough.

tempfile 10mo ago
I think the evidence is actually pretty strongly against them doing anything similar to "thinking". Certainly they are exhibiting some behaviour that we have traditionally only associated with thinking. But this comes along with lots of behaviour that is fundamentally opposite to thinking ("hallucination" being the major example). It seems much more likely that they are doing some other behaviour that only sometimes resembles thinking, in the same way that when you press the middle autocomplete button on your phone keyboard it only sometimes resembles conversation.
- groestl 10mo ago
  > "hallucination" ... "behaviour that only sometimes resembles thinking" I guess you'll find that if you limit the definition of thinking that much most humans are not capable of thinking either.
  - tempfile 10mo ago
    I have never in my life met a person who hallucinates in the way ChatGPT etc do. If I did, I would probably assume they were deliberately lying, or very unwell.
  - Yizahi 10mo ago
    You see, we are here observing a clash in the terminology. Hallucinations in humans is thinking, just not typical. So called "hallucinations" in LLM programs are just noise output, a garbage. This is why using anthropomorphic terms for programs is bad. Just like "thinking" or "reasoning".

bonplan23 10mo ago

Nice that LLMs can now argue to defend themselves. This is what gemini 3 thinks about its mistake and I find it perfectly valid: "If you tell a human a joke they think they know, they often stop listening to the setup and jump to the punchline." And when we say "stop listing" we don't actually mean that they shut their ears, but that they activate a once established neural shortcut - just as the LLM did.

ffwd 10mo ago

LLMs _can_ think top-to-bottom but only if you make them think about concrete symbol based problems. Like this one: https://chatgpt.com/s/t_692d55a38e2c8191a942ef2689eb4f5a https://chatgpt.com/s/t_692d55a38e2c8191a942ef2689eb4f5a The prompt I used was "write out the character 'R' in ascii art using exactly 62 # for the R and 91 Q characters to surround it with" Here it has a top down goal of keeping the exact amount of #'s and Q's and it does keep it in the output. The purpose of this is to make it produce the asciii art in a step by step manner instead of fetching a premade ascii art from training data. What it does not reason well about always are abstract problems like the doctor example in the post. The real key for reasoning IMO is the ability to decompose the text into a set of components, then apply world model knowledge to those components, then having the ability to manipulate those components based on what they represent. Humans have an associative memory so when we read a word like "doctor", our brain gathers the world knowledge about that word automatically. It's kind of hard to tell exactly what world knowledge the LLM has vs doesn't have, but it seems like it's doing some kind of segmentation of words, sentences and paragraphs based on the likelihood of those patterns in the training data, and then it can do _some_ manipulation on those patterns based on other likelihood of those patterns. Like for example if there is a lot of text talking about what a doctor is, then that produces a probability distribution about what a doctor is, which it then can use in other prompts relating to doctors. But I have seen this fail before as all of this knowledge is not combined into one world model but rather purely based on the prompt and the probabilities associated with that prompt. It can contradict itself in other words.

zerosizedweasle 10mo ago

oldestofsports 10mo ago

Please note that chatgpt is unable to play chess

Yizahi 10mo ago

No they don't. When queried how exactly did a program arrive to a specific output it will happily produce some output resembling thinking and having all the required human-like terminology. The problem is that it doesn't match at all with how the LLM program calculated output in reality. So the "thinking" steps are just a more of the generated BS, to fool us more. One point to think about - an entity being tested for intelligence/thinking/etc only needs to fail once, o prove that it is not thinking. While the reverse applies too - to prove that a program is thinking it must be done in 100% of tests, or the result is failure. And we all know many cases when LLMs are clearly not thinking, just like in my example above. So the case is rather clear for the current gen of LLMs.

roxolotl 10mo ago
This is an interesting point but while I agree with the article, don’t think LLMs are more than sophisticated autocomplete, and believe there’s way more to human intelligence than matrix multiplication humans also cannot explain in many cases why they did what they did. Of course the most famous and clear example are the split brain experiments which show post hoc rationalization[0]. And then there’s the Libet experiments[1] showing that your conscious experience is only realized after the triggering brain activity. While it’s not showing you cannot explain why it does seem to indicate your explanation is post hoc. 0: https://www.neuroscienceof.com/human-nature-blog/decision-making-split-brain https://www.neuroscienceof.com/human-nature-blog/decision-ma... 1: https://www.informationphilosopher.com/freedom/libet_experiments.html https://www.informationphilosopher.com/freedom/libet_experim...
- Yizahi 10mo ago
  I agree, but here we are veering into more complex decision making. I was talking about much simpler cases, like for example going through a handful of simple steeps for simple task. For example addition, ask a person to sum two numbers and then ask to explain what he just did step by step and a person would be able to do it. Person may even make a mistake in process but the general algorithm will be matching what actually happened. Query LLM for the same, and while LLM answer will be correct for a human, it won't match what LLM actually did to calculate. This is what outs LLM "thinking" for me, they just generate a very plausible intermediate steps too.

greatgib 10mo ago

A real debate is possible on the subject but this blog post worth nothing on the subject. From my side, I don't really know if what does LLM is thinking, but what amaze me is that: - It is clear to me the way the LLM operate that things are generated token after token, without really a pre-existing plan on what comes next. So, more like a probabilistic repeating machine. - But it the same time, I can see in action LLM capable to create things or reply to questions that clearly does not exist in the training corpus. So it shows a behavior that is similar to thinking to complete tasks. For example, let's suppose you give him specific tools to your own custom API, you ask him to do a task, and we can observe that it is capable of mixing multiple calls and combination of the tools results to achieve a given purpose. Otherwise, when you ask LLM to do math operations like 3123454*2030+500 and it is capable to give the good reply (not all the cases but sometimes). Where, despite the huge size of the corpus, there is not exactly all the operations that are exactly available in the corpus for sure. So, my best guess is that a lot of things in our world are based on "semantic" patterns that we don't know. Especially for math and logic that are bound to the language. To me it is similar to the mentral trick used by "fast calculator".

josefritzishere 10mo ago

It seems that Betteridge's law of headlines applies here.

hkkwritesgmail 10mo ago

If the AI truly becomes human like, then it should also make mistakes like a human does even on simple arithmetic. Now, AI might make mistakes on simple arithmetic just like humans. However, once the human is given a hint on the fact that there is some simple arithmetic mistake to be corrected without any details, then the human corrects it. But the AI never does.

IAmBroom 10mo ago
You're moving the goalposts. The article asks if AI thinks, not if AI thinks just like humans do.