5 ms·

No it really isn't. The Turing test is just not an adequate methodology to determine intelligent behavior, and never was. This was already known way before gene

No it really isn't. The Turing test is just not an adequate methodology to determine intelligent behavior, and never was. This was already known way before generative ML models emerged. https://en.wikipedia.org/wiki/Turing_test#Weaknesses https://en.wikipedia.org/wiki/Turing_test#Weaknesses Just to pick my personal favorite, which is mentioned at the end of the article: This function right here can, technically, pass the Turing-Test: def generate_answer(in: str) -> str: return "" How? Simple: A human can just chose not to respond to any question. So, if a program does exactly that, that is, meet every query with silence, how do you differentiate it from a human who does the same?

WendyTheWillow 3y ago
That’s an absurd and overly strict interpretation of what Turing described. Stipulating cooperation between participants is precisely in the spirit of Turing’s original work.
- usrbinbash 3y ago
  > That’s an absurd and overly strict interpretation of what Turing described. No, it isn't. The turing test doesn't evaluate correctness of any answers, their sophistication, or even if there is an answer. All it evaluates is the ability of the interrogator to distinguish between the computer and the human. And therein lies the greatest flaw of the test: It doesn't test the ability of the computer, it tests the ability of the interrogator. Quote from the wikipedia article: https://en.wikipedia.org/wiki/Turing_test# https://en.wikipedia.org/wiki/Turing_test# In practice, the test's results can easily be dominated not by the computer's intelligence, but by the attitudes, skill, or naïveté of the questioner. Numerous experts in the field, including cognitive scientist Gary Marcus, insist that the Turing test only shows how easy it is to fool humans and is not an indication of machine intelligence. And another quote: Chatterbot programs such as ELIZA have repeatedly fooled unsuspecting people into believing that they are communicating with human beings. In these cases, the "interrogators" are not even aware of the possibility that they are interacting with computers. To successfully appear human, there is no need for the machine to have any intelligence whatsoever and only a superficial resemblance to human behaviour is required So the "silence program" may be an extreme case, but it showcases exactly this. If the computer simply says nothing, then what can the human do to determine it's a computer who is silent behind the curtain? And the answer is: Nothing. He can only guess. And since a person can just as easily be silent as a computer can, he might even mistake the human performer for a computer.
  - WendyTheWillow 3y ago
    Yes, it's an objectively wrong interpretation of Turing's Imitation Game outlined in his paper, "Computing Machinery and Intelligence", published in Mind in 1940 [0]. It's literally on the first page: > Now suppose X is actually A, then A must answer. It is A's object in the game to try and cause C to make the wrong identification. A must answer. [0] https://redirect.cs.umbc.edu/courses/471/papers/turing.pdf https://redirect.cs.umbc.edu/courses/471/papers/turing.pdf
    - usrbinbash 3y ago
      > A must answer Oh really? https://www.researchgate.net/publication/289523532_Taking_the_fifth_amendment_in_Turing's_imitation_game https://www.researchgate.net/publication/289523532_Taking_th...
      - WendyTheWillow 3y ago
        Yes, really. > practical Turing tests Practical Turing tests. Edit: Here's the justification your paper uses to suggest the idea that Turing meant for the possibility of silence as a response: > In one interpretation of Turing’s test the female is expected to tell the truth, but we are not far off that time when silence was preferred to the “jabbering” of women, because “speech was the monopoly of man” and that “sounds made by birds were part of a conversation at least as intelligible and intelligent as the confusion of tongues arising at a fashionable lady’s reception”. Additionally, your cited paper there even admits this is a theoretical extension of The Imitation Game: > In its standard form, Turing’s imitation game is described as an experiment that can be practicalized in two different ways (see Figure 1) (Shah, 2011): 1) one-interrogator-one hidden interlocutor (Figure 1a), 2) one-interrogator-two hidden interlocutors (Figure 1b). > In both cases the machine must provide “satisfactory” and “sustained” answers to any questions put to it by the human interrogator (Turing, 1950: p.447). However, what about in the theoretical case when the machine takes the 5th amendment: “No person shall be held to answer”?1 Would we grant “fair play to the machines”? To repeat in case you missed it when you clearly and definitely read your own citation: "In both cases the machine must provide “satisfactory” and “sustained” answers to any questions put to it by the human interrogator (Turing, 1950: p.447)."
paulddraper 3y ago
> how do you differentiate it from a human who does the same You're begging the question. If a human gives the same answers, you can't differentiate. But a human would not give the same answer (empty string) as your function.
- usrbinbash 3y ago
  > But a human would not give the same answer (empty string) as your function. He doesn't have to. The simple fact that a human CAN do that is enough to make the determination in this case guesswork.
  - paulddraper 3y ago
    A human could respond in only Monty Python quotes. But that's not realistic. --- The Turing test is determined by % of people who believe they are taking to a person.
    - usrbinbash 3y ago
      > The Turing test is determined What does the Turing Test determine anyway? According to the paper, it is supposed to measure a machines intelligence, or rather more prosaically, answer the question "Can machines think?" But that isn't what the test measures. It measures how well a machine can trick a human into believing it is a person. So instead of measuring how well the machine does, the test instead measures how well the human does. That is the greatest flaw of the Turing Test, and the little "answer with silence" thought experiment is showcasing exactly that flaw. "Can a machine trick a human" and "Can a machine think" are 2 very different questions. Humans can, and have shown to, be tricked by ELIZA and even simpler chatterbots, engines that don't even use any kind of ML, just large bodies of prewritten text and a number of static rules. > by % of people who believe they are taking to a person. And what does that denote? Say I get 2 groups of people. One is tricked by ELIZA 80% of the time, the other is tricked by ELIZA 40% of the time. Does that show that ELIZA passed or failed? Neither. It shows that the outcome of the test depends as much, or even more, on the ability of the interrogator than it does on the quality of the machines responses. Imagine a litmus test (a chemical test to roughly determine the acidity of a solution), where the test result depends on who performs it, as much or even more than it does on the quality of the Litmus-Paper. No lab would use that test for obvious reasons.
GaggiX 3y ago
Okay so Turing test but there is an actual conversation (I doubt in actual implementations like the one linked by OP the bot or the human can send empty messages) the Turing test is an adequate methodology?

WendyTheWillow 3y ago

That’s an absurd and overly strict interpretation of what Turing described. Stipulating cooperation between participants is precisely in the spirit of Turing’s original work.

usrbinbash 3y ago
> That’s an absurd and overly strict interpretation of what Turing described. No, it isn't. The turing test doesn't evaluate correctness of any answers, their sophistication, or even if there is an answer. All it evaluates is the ability of the interrogator to distinguish between the computer and the human. And therein lies the greatest flaw of the test: It doesn't test the ability of the computer, it tests the ability of the interrogator. Quote from the wikipedia article: https://en.wikipedia.org/wiki/Turing_test# https://en.wikipedia.org/wiki/Turing_test# In practice, the test's results can easily be dominated not by the computer's intelligence, but by the attitudes, skill, or naïveté of the questioner. Numerous experts in the field, including cognitive scientist Gary Marcus, insist that the Turing test only shows how easy it is to fool humans and is not an indication of machine intelligence. And another quote: Chatterbot programs such as ELIZA have repeatedly fooled unsuspecting people into believing that they are communicating with human beings. In these cases, the "interrogators" are not even aware of the possibility that they are interacting with computers. To successfully appear human, there is no need for the machine to have any intelligence whatsoever and only a superficial resemblance to human behaviour is required So the "silence program" may be an extreme case, but it showcases exactly this. If the computer simply says nothing, then what can the human do to determine it's a computer who is silent behind the curtain? And the answer is: Nothing. He can only guess. And since a person can just as easily be silent as a computer can, he might even mistake the human performer for a computer.
- WendyTheWillow 3y ago
  Yes, it's an objectively wrong interpretation of Turing's Imitation Game outlined in his paper, "Computing Machinery and Intelligence", published in Mind in 1940 [0]. It's literally on the first page: > Now suppose X is actually A, then A must answer. It is A's object in the game to try and cause C to make the wrong identification. A must answer. [0] https://redirect.cs.umbc.edu/courses/471/papers/turing.pdf https://redirect.cs.umbc.edu/courses/471/papers/turing.pdf
  - usrbinbash 3y ago
    > A must answer Oh really? https://www.researchgate.net/publication/289523532_Taking_the_fifth_amendment_in_Turing's_imitation_game https://www.researchgate.net/publication/289523532_Taking_th...
    - WendyTheWillow 3y ago
      Yes, really. > practical Turing tests Practical Turing tests. Edit: Here's the justification your paper uses to suggest the idea that Turing meant for the possibility of silence as a response: > In one interpretation of Turing’s test the female is expected to tell the truth, but we are not far off that time when silence was preferred to the “jabbering” of women, because “speech was the monopoly of man” and that “sounds made by birds were part of a conversation at least as intelligible and intelligent as the confusion of tongues arising at a fashionable lady’s reception”. Additionally, your cited paper there even admits this is a theoretical extension of The Imitation Game: > In its standard form, Turing’s imitation game is described as an experiment that can be practicalized in two different ways (see Figure 1) (Shah, 2011): 1) one-interrogator-one hidden interlocutor (Figure 1a), 2) one-interrogator-two hidden interlocutors (Figure 1b). > In both cases the machine must provide “satisfactory” and “sustained” answers to any questions put to it by the human interrogator (Turing, 1950: p.447). However, what about in the theoretical case when the machine takes the 5th amendment: “No person shall be held to answer”?1 Would we grant “fair play to the machines”? To repeat in case you missed it when you clearly and definitely read your own citation: "In both cases the machine must provide “satisfactory” and “sustained” answers to any questions put to it by the human interrogator (Turing, 1950: p.447)."

paulddraper 3y ago

> how do you differentiate it from a human who does the same You're begging the question. If a human gives the same answers, you can't differentiate. But a human would not give the same answer (empty string) as your function.

usrbinbash 3y ago
> But a human would not give the same answer (empty string) as your function. He doesn't have to. The simple fact that a human CAN do that is enough to make the determination in this case guesswork.
- paulddraper 3y ago
  A human could respond in only Monty Python quotes. But that's not realistic. --- The Turing test is determined by % of people who believe they are taking to a person.
  - usrbinbash 3y ago
    > The Turing test is determined What does the Turing Test determine anyway? According to the paper, it is supposed to measure a machines intelligence, or rather more prosaically, answer the question "Can machines think?" But that isn't what the test measures. It measures how well a machine can trick a human into believing it is a person. So instead of measuring how well the machine does, the test instead measures how well the human does. That is the greatest flaw of the Turing Test, and the little "answer with silence" thought experiment is showcasing exactly that flaw. "Can a machine trick a human" and "Can a machine think" are 2 very different questions. Humans can, and have shown to, be tricked by ELIZA and even simpler chatterbots, engines that don't even use any kind of ML, just large bodies of prewritten text and a number of static rules. > by % of people who believe they are taking to a person. And what does that denote? Say I get 2 groups of people. One is tricked by ELIZA 80% of the time, the other is tricked by ELIZA 40% of the time. Does that show that ELIZA passed or failed? Neither. It shows that the outcome of the test depends as much, or even more, on the ability of the interrogator than it does on the quality of the machines responses. Imagine a litmus test (a chemical test to roughly determine the acidity of a solution), where the test result depends on who performs it, as much or even more than it does on the quality of the Litmus-Paper. No lab would use that test for obvious reasons.

GaggiX 3y ago

Okay so Turing test but there is an actual conversation (I doubt in actual implementations like the one linked by OP the bot or the human can send empty messages) the Turing test is an adequate methodology?