6 ms·

A ridiculous argument. Turing machines don't know anything about the program they are executing. In fact, Turing machines don't "know" anything. Turing machines

A ridiculous argument. Turing machines don't know anything about the program they are executing. In fact, Turing machines don't "know" anything. Turing machines don't know how to fly a plane, translate a language, or play chess. The program does. And Searle puts the man in the room in the place of the Turing machine.

wk_end 11mo ago
So what, in the analogy, would be the program? Surely it's not the printed rules, so I think you're making the "systems reply" - that the program that knows Chinese is some sort of metaphysical "system" that arises from the man using the rules - which is the first thing Searle tries to rebut. > let the individual internalize all of these elements of the system. He memorizes the rules in the ledger and the data banks of Chinese symbols, and he does all the calculations in his head. The individual then incorporates the entire system. There isn't anything at all to the system that he does not encompass. We can even get rid of the room and suppose he works outdoors. All the same, he understands nothing of the Chinese, and a fortiori neither does the system, because there isn't anything in the system that isn't in him. If he doesn't understand, then there is no way the system could understand because the system is just a part of him. In other words, even if you put the man in place of everything, there's still a gap between mechanically manipulating symbols and actual understanding.
- BadThink6655321 11mo ago
  Only because "actual understanding" is ambiguously defined. Meaning is an association of A with B. Our brains have a large associative array with the symbols for the sound "dog" is associated with the image of "dog' which is associated with the behavior of "dog" which is associated with the feel of "dog", ... We associate the symbols for the word "hamburger" with the symbols for the taste of "hamburger", with ... We undersand something when our past associations match current inputs and can predict furture inputs.
  - siglesias 11mo ago
    "Actual understanding" means you have a grounding for the word down to conscious experience and you have a sense of certainty about its associations. I don't understand "sweetness" because I competently use the word "sweet." I understand sweetness because I have a sense of it all the way down to the experience of sweetness AND the natural positive associations and feelings I have with it. There HAS to be some distinction between understanding all the way down to sensation and a competent or convincing deployment of that symbol without those sensations. If we think about how we "train" AI to "understand" sweetness, we're basically telling it when and when not to use that symbol in the context of other symbols (or visual inputs). We don't do this when we teach a child that word. The child has an inner experience he can associate with other tastes.
    - deleted 11mo ago
      [deleted]
    - mannykannot 11mo ago
      The irony here is that performing like an LLM the very thing that Searle has the human operator do. If it the sort of interaction that does not need intelligence, then no conclusion about the feasibility of AGI can be drawn from contemplating it. Searle’s arguments have been overtaken by technology.
      - siglesias 11mo ago
        Can you expand on this? The thought experiment is just about showing that there is more to having a mind than having a program. It’s not an argument about the capabilities of LLMs or AGI. Though it’s worth noting that behavioral criteria continue to lead to people overestimating the capabilities of promise of AI.
        mannykannot 11mo ago
        LLMs are capable of performing the task specified for the Chinese room over a wide range of complex topics and for a considerable length of time. While it is true that their productions are wrong or ill-conceived more often than one would expect from a well-informed human, and sometimes look like the work of a rather stupid one, the burden now rests on Searle's successors to show that every such interaction is purely syntactic.
- mannykannot 11mo ago
  People are doing things they personally do not understand, just by following the rules, all the time. One does not need to understand why celestial navigation works in order to do it, for example. Heck, most kids can learn arithmetic (and perform it in their heads) without being able to explain why it works, and many (including their teachers, sometimes) never achieve that understanding. Searle’s failure to recognize this very real possibility amounts to tacit question-begging.
  - TheOtherHobbes 11mo ago
    Yes, it's a wrong-end-of-the-telescope kind of answer. A human does simulates a Turing machine to do... something. The human is acting mechanically. So what? If there's any meaning, it exists outside the machine and the human simulating it. You need another human to understand the results. All Searle has done is distract everyone from whatever is going on inside that other human.
- alt-normal 11mo ago
  [dead]
- rcxdude 11mo ago
  In that case you've basically just created a split-brain situation (I mean like the actual phenomenon of someone who's had the main part of the connection between the two hemispheres of the brain). There's one system which is the man and the rules that he has internalized, and there's what the man himself consciously understands, and there's no reason that the two are necessarily communicating in some deeper way, in much the same way as a split-brain patient may be able to point to something they see in one side of their vision when asked but be unable to say what it is. (Also, IMO, the question of whether the program understands chinese mainly depends on whether you would describe an unconscious person as understanding anything) I also can't help but think of this sketch when this topic comes up (even though, importantly, it is not quite the same thing): https://www.youtube.com/watch?v=6vgoEhsJORU https://www.youtube.com/watch?v=6vgoEhsJORU
- glyco 11mo ago
  You and Searle both seem to not understand a simple, obvious fact about the world, which is that (inhomogenous) things don't have the same thing inside. A chicken pie, for example, doesn't have any chicken pie inside. There's chicken inside, but that's not chicken pie. There's sauce, vegetables and pastry, but those aren't chicken pie either. All these things together still may not make a chicken pie. The 'chickenpieness' of the pie is an additional fact, not derivable from any facts about its components. As with pie, so with 'understanding'. A system which understands can be expected to not contain anything which understands. So if you find a system which contains nothing which understands, this tells you nothing about whether the system understands[0]. Somehow both you and Searle have managed to find this simple fact about pie to be 'the grip of an ideology' and 'metaphysical'. But it really isn't. [0] And vice-versa, as in Searle's pointlessly overcomplicated example of a system which understands Chinese containing one which doesn't containing one which does.
deleted 11mo ago
[deleted]

wk_end 11mo ago

So what, in the analogy, would be the program? Surely it's not the printed rules, so I think you're making the "systems reply" - that the program that knows Chinese is some sort of metaphysical "system" that arises from the man using the rules - which is the first thing Searle tries to rebut. > let the individual internalize all of these elements of the system. He memorizes the rules in the ledger and the data banks of Chinese symbols, and he does all the calculations in his head. The individual then incorporates the entire system. There isn't anything at all to the system that he does not encompass. We can even get rid of the room and suppose he works outdoors. All the same, he understands nothing of the Chinese, and a fortiori neither does the system, because there isn't anything in the system that isn't in him. If he doesn't understand, then there is no way the system could understand because the system is just a part of him. In other words, even if you put the man in place of everything, there's still a gap between mechanically manipulating symbols and actual understanding.

BadThink6655321 11mo ago
Only because "actual understanding" is ambiguously defined. Meaning is an association of A with B. Our brains have a large associative array with the symbols for the sound "dog" is associated with the image of "dog' which is associated with the behavior of "dog" which is associated with the feel of "dog", ... We associate the symbols for the word "hamburger" with the symbols for the taste of "hamburger", with ... We undersand something when our past associations match current inputs and can predict furture inputs.
- siglesias 11mo ago
  "Actual understanding" means you have a grounding for the word down to conscious experience and you have a sense of certainty about its associations. I don't understand "sweetness" because I competently use the word "sweet." I understand sweetness because I have a sense of it all the way down to the experience of sweetness AND the natural positive associations and feelings I have with it. There HAS to be some distinction between understanding all the way down to sensation and a competent or convincing deployment of that symbol without those sensations. If we think about how we "train" AI to "understand" sweetness, we're basically telling it when and when not to use that symbol in the context of other symbols (or visual inputs). We don't do this when we teach a child that word. The child has an inner experience he can associate with other tastes.
  - deleted 11mo ago
    [deleted]
  - mannykannot 11mo ago
    The irony here is that performing like an LLM the very thing that Searle has the human operator do. If it the sort of interaction that does not need intelligence, then no conclusion about the feasibility of AGI can be drawn from contemplating it. Searle’s arguments have been overtaken by technology.
    - siglesias 11mo ago
      Can you expand on this? The thought experiment is just about showing that there is more to having a mind than having a program. It’s not an argument about the capabilities of LLMs or AGI. Though it’s worth noting that behavioral criteria continue to lead to people overestimating the capabilities of promise of AI.
      - mannykannot 11mo ago
        LLMs are capable of performing the task specified for the Chinese room over a wide range of complex topics and for a considerable length of time. While it is true that their productions are wrong or ill-conceived more often than one would expect from a well-informed human, and sometimes look like the work of a rather stupid one, the burden now rests on Searle's successors to show that every such interaction is purely syntactic.
mannykannot 11mo ago
People are doing things they personally do not understand, just by following the rules, all the time. One does not need to understand why celestial navigation works in order to do it, for example. Heck, most kids can learn arithmetic (and perform it in their heads) without being able to explain why it works, and many (including their teachers, sometimes) never achieve that understanding. Searle’s failure to recognize this very real possibility amounts to tacit question-begging.
- TheOtherHobbes 11mo ago
  Yes, it's a wrong-end-of-the-telescope kind of answer. A human does simulates a Turing machine to do... something. The human is acting mechanically. So what? If there's any meaning, it exists outside the machine and the human simulating it. You need another human to understand the results. All Searle has done is distract everyone from whatever is going on inside that other human.
alt-normal 11mo ago
[dead]
rcxdude 11mo ago
In that case you've basically just created a split-brain situation (I mean like the actual phenomenon of someone who's had the main part of the connection between the two hemispheres of the brain). There's one system which is the man and the rules that he has internalized, and there's what the man himself consciously understands, and there's no reason that the two are necessarily communicating in some deeper way, in much the same way as a split-brain patient may be able to point to something they see in one side of their vision when asked but be unable to say what it is. (Also, IMO, the question of whether the program understands chinese mainly depends on whether you would describe an unconscious person as understanding anything) I also can't help but think of this sketch when this topic comes up (even though, importantly, it is not quite the same thing): https://www.youtube.com/watch?v=6vgoEhsJORU https://www.youtube.com/watch?v=6vgoEhsJORU
glyco 11mo ago
You and Searle both seem to not understand a simple, obvious fact about the world, which is that (inhomogenous) things don't have the same thing inside. A chicken pie, for example, doesn't have any chicken pie inside. There's chicken inside, but that's not chicken pie. There's sauce, vegetables and pastry, but those aren't chicken pie either. All these things together still may not make a chicken pie. The 'chickenpieness' of the pie is an additional fact, not derivable from any facts about its components. As with pie, so with 'understanding'. A system which understands can be expected to not contain anything which understands. So if you find a system which contains nothing which understands, this tells you nothing about whether the system understands[0]. Somehow both you and Searle have managed to find this simple fact about pie to be 'the grip of an ideology' and 'metaphysical'. But it really isn't. [0] And vice-versa, as in Searle's pointlessly overcomplicated example of a system which understands Chinese containing one which doesn't containing one which does.

deleted 11mo ago

[deleted]