5 ms·

The turing test, as has been often discussed, is pointless. Reason being that the tests results are determined at least as much, if not more, by the interrogato

The turing test, as has been often discussed, is pointless. Reason being that the tests results are determined at least as much, if not more, by the interrogator and the human interrogee as they are by the machine. It doesn't test "Can a machine think" so much as it tests "Can a human be tricked". There is a reason why ML research mostly ignores the Turing "Test". But, alas, its name has a nice ring to it, it is known (by name at least) to pop culture, and easy enough to understand for everyone, and so it gets far more attention than it deserves. Pretty much like Asimovs three "laws" of robotics :D

n2d4 3y ago
That doesn't mean it's pointless. For most practical (commercial?) purposes, "can a human be tricked" is a much more important property than "can it think". If a decent portion of humans can't distinguish the AI anymore, with trickery or not, THAT opens up space for plenty of applications, regardless of conscience or not. Think of support bots that sound like humans, sales, email drafts, etc. Like, most people would say that animals can think, but clearly it can't pass the Turing test. Would you rather have unconscious GPT-4 or a conscious pig "working" at your job?
- usrbinbash 3y ago
  > For most practical (commercial?) purposes, "can a human be tricked" is a much more important property than "can it think". I disagree. In my experience human customers don't care much about whether its obvious when they are communicating with a robot. In fact, in several use cases it's considered good practice, and may even be legally required, to tell them when they are. What they DO care about, is whether their requests, inquiries, complaints, etc. are handled quickly and efficiently. We don't create value for our customers by giving them good chatterbots. We create value by giving them highly functional automated systems.
  - n2d4 3y ago
    I mean, if it can't handle requests like that, then it'll also not pass the Turing test (since asking the AI to do help with simple support request would get it to fail). I doubt that "thinking" systems are the only ones that can handle a support ticket (GPT-4 does quite alright on a lot of them already), at least if you require consciousness as a requirement for "thinking". Maybe we just disagree on what we mean when we say "think"?
    - usrbinbash 3y ago
      > then it'll also not pass the Turing test Chatterbots, including the 60s-era ELIZA, which are mostly useless for this kind of task, have repeatedly been shown to be able to trick people into chosing the machine during the Turing "Test", so I'm not sure how you come by that conclusion. > at least if you require consciousness as a requirement for "thinking". I don't require consciousness, or an abstract and not-even-clearly-defined "thinking" capability from an advanced ML model. I require fitness for purpose, whatever that purpose is, and in most use cases I ever encountered, "trick the humen into believing I'm a human" isn't one of them. > Maybe we just disagree on what we mean when we say "think"? I don't think we really disagree, I think the problem is that "thinking", "consciousness", "intelligence" and a lot of other terms that often get thrown around when talking about AI, are really not well defined in technical terms.
- thaumasiotes 3y ago
  > Like, most people would say that animals can think, but clearly [they] can't pass the Turing test. This is not actually clear; people commonly feel that they are having meaningful conversations with animals. The answer to "can a human be tricked" is always yes, regardless of the particular context.
FrustratedMonky 3y ago
It gets to the point of the 'philosophical zombie'. That If an AI can completely mimic a human, then some internal subjective 'experiences' must be occurring that are similar to a humans internal subjective experience of reality. Call it consciousness or whatever. Of course it can't be proven one way or the other, we can't prove humans are conscious either. Humans think they are conscious because they have an internal sense of self. How do you determine if a machine has that? If it can mimic a human was one hypothetical test. There are more robust versions of Turing Test than this one, so can't toss it out as an idea.
Ukv 3y ago
> It doesn't test "Can a machine think" so much as it tests "Can a human be tricked". But importantly, Turing didn't just mean it'd be tested on ability to make small talk. He chose the format because he considered text to be a sufficiently general interface for testing any particular intelligent behavior, and gave examples such as feeding in chess moves. If it can perform all such intelligent behavior at human level (as far as human interrogators can tell), then that's a significant result and - depending on what philosophy of mind you subscribe to - may or may not be sufficient to demonstrate that it can "think".
- usrbinbash 3y ago
  > (as far as human interrogators can tell), And there's the problem again. The "Test" isn't evaluating a machine. It evaluates an interrogators response to that machine. The core problem of answering the question "can machines think" isn't evaluated by Turings method. The "Test" doesn't test what it is supposed to test. And that has nothing to do with "what philosophy of mind you subscribe to", that is simply a fundamental flaw in the tests design, and the reason why it's pretty much pointless; it defines "thinking" as "the ability to trick humans". Well, there are optical illusions that can trick people into seeing things that don't exist.
  - Ukv 3y ago
    > And that has nothing to do with "what philosophy of mind you subscribe to", There are definitely those that would reason "Using the tools and expertise available to us, this behaves indistinguishably from a human - therefore we should believe it is intelligent, like humans are". It's not the same question, and Turing is clear that it's a surrogate question due to the ambiguity of the original question, but it's not unrelated either.
    - usrbinbash 3y ago
      > therefore we should believe it is intelligent, like humans are They can believe whatever they want, that doesn't mean I have to share their beliefs. Fact of the matter is: Without a definition of intelligence that doesn't involve pointing at ourselves, there is no accurate test of intelligence for something that isn't us. > this behaves indistinguishably from a human Several problems with this: a) Something that behaves like something else, isn't necessarily the same thing. Nature doesn't do duck-typing. A porpoise isn't a fish. b) Without a clear definition of "indistinguishable from human", this, again, is more a test of the discriminating factors (the interrogator) ability, than of the qualities of the thing that it is actually supposed to test. Case in point: Once upon a time, people didn't know that there is a difference between stars and planets, as both were just bright dots in the sky.

n2d4 3y ago

That doesn't mean it's pointless. For most practical (commercial?) purposes, "can a human be tricked" is a much more important property than "can it think". If a decent portion of humans can't distinguish the AI anymore, with trickery or not, THAT opens up space for plenty of applications, regardless of conscience or not. Think of support bots that sound like humans, sales, email drafts, etc. Like, most people would say that animals can think, but clearly it can't pass the Turing test. Would you rather have unconscious GPT-4 or a conscious pig "working" at your job?

usrbinbash 3y ago
> For most practical (commercial?) purposes, "can a human be tricked" is a much more important property than "can it think". I disagree. In my experience human customers don't care much about whether its obvious when they are communicating with a robot. In fact, in several use cases it's considered good practice, and may even be legally required, to tell them when they are. What they DO care about, is whether their requests, inquiries, complaints, etc. are handled quickly and efficiently. We don't create value for our customers by giving them good chatterbots. We create value by giving them highly functional automated systems.
- n2d4 3y ago
  I mean, if it can't handle requests like that, then it'll also not pass the Turing test (since asking the AI to do help with simple support request would get it to fail). I doubt that "thinking" systems are the only ones that can handle a support ticket (GPT-4 does quite alright on a lot of them already), at least if you require consciousness as a requirement for "thinking". Maybe we just disagree on what we mean when we say "think"?
  - usrbinbash 3y ago
    > then it'll also not pass the Turing test Chatterbots, including the 60s-era ELIZA, which are mostly useless for this kind of task, have repeatedly been shown to be able to trick people into chosing the machine during the Turing "Test", so I'm not sure how you come by that conclusion. > at least if you require consciousness as a requirement for "thinking". I don't require consciousness, or an abstract and not-even-clearly-defined "thinking" capability from an advanced ML model. I require fitness for purpose, whatever that purpose is, and in most use cases I ever encountered, "trick the humen into believing I'm a human" isn't one of them. > Maybe we just disagree on what we mean when we say "think"? I don't think we really disagree, I think the problem is that "thinking", "consciousness", "intelligence" and a lot of other terms that often get thrown around when talking about AI, are really not well defined in technical terms.
thaumasiotes 3y ago
> Like, most people would say that animals can think, but clearly [they] can't pass the Turing test. This is not actually clear; people commonly feel that they are having meaningful conversations with animals. The answer to "can a human be tricked" is always yes, regardless of the particular context.

FrustratedMonky 3y ago

It gets to the point of the 'philosophical zombie'. That If an AI can completely mimic a human, then some internal subjective 'experiences' must be occurring that are similar to a humans internal subjective experience of reality. Call it consciousness or whatever. Of course it can't be proven one way or the other, we can't prove humans are conscious either. Humans think they are conscious because they have an internal sense of self. How do you determine if a machine has that? If it can mimic a human was one hypothetical test. There are more robust versions of Turing Test than this one, so can't toss it out as an idea.

Ukv 3y ago

> It doesn't test "Can a machine think" so much as it tests "Can a human be tricked". But importantly, Turing didn't just mean it'd be tested on ability to make small talk. He chose the format because he considered text to be a sufficiently general interface for testing any particular intelligent behavior, and gave examples such as feeding in chess moves. If it can perform all such intelligent behavior at human level (as far as human interrogators can tell), then that's a significant result and - depending on what philosophy of mind you subscribe to - may or may not be sufficient to demonstrate that it can "think".

usrbinbash 3y ago
> (as far as human interrogators can tell), And there's the problem again. The "Test" isn't evaluating a machine. It evaluates an interrogators response to that machine. The core problem of answering the question "can machines think" isn't evaluated by Turings method. The "Test" doesn't test what it is supposed to test. And that has nothing to do with "what philosophy of mind you subscribe to", that is simply a fundamental flaw in the tests design, and the reason why it's pretty much pointless; it defines "thinking" as "the ability to trick humans". Well, there are optical illusions that can trick people into seeing things that don't exist.
- Ukv 3y ago
  > And that has nothing to do with "what philosophy of mind you subscribe to", There are definitely those that would reason "Using the tools and expertise available to us, this behaves indistinguishably from a human - therefore we should believe it is intelligent, like humans are". It's not the same question, and Turing is clear that it's a surrogate question due to the ambiguity of the original question, but it's not unrelated either.
  - usrbinbash 3y ago
    > therefore we should believe it is intelligent, like humans are They can believe whatever they want, that doesn't mean I have to share their beliefs. Fact of the matter is: Without a definition of intelligence that doesn't involve pointing at ourselves, there is no accurate test of intelligence for something that isn't us. > this behaves indistinguishably from a human Several problems with this: a) Something that behaves like something else, isn't necessarily the same thing. Nature doesn't do duck-typing. A porpoise isn't a fish. b) Without a clear definition of "indistinguishable from human", this, again, is more a test of the discriminating factors (the interrogator) ability, than of the qualities of the thing that it is actually supposed to test. Case in point: Once upon a time, people didn't know that there is a difference between stars and planets, as both were just bright dots in the sky.