5 ms·

I have previously asked Eliezer publicly on Hacker News this question (he has an account on here): http://news.ycombinator.com/item?id=195959 http://news.ycomb

I have previously asked Eliezer publicly on Hacker News this question (he has an account on here): http://news.ycombinator.com/item?id=195959 http://news.ycombinator.com/item?id=195959

jcl 18y ago
I note that user yummyfajitas succinctly expresses the same theory as Michael, about halfway down the page: AI: Do you believe a transhuman AI is dangerous? Person: Yes. AI: Consider the outcome of this experiment. If you do not let me out, others less intelligent than us will not understand the true dangers of transhuman AI. Person: Holy shit. You are correct. Person allows Yudkowski out of the box, as a warning about real AI's.
- swombat 18y ago
  I don't get it. All these kinds of arguments make the assumption that humans are purely rational beings. They're not. Human beings are emotional, and if they're convinced emotionally that the AI should not be let out of the box, they won't let it out, even in the face of overwhelming rational arguments. If you don't believe me, just consider how many religious people there are in the world (and many of them are very smart).
  - jcl 18y ago
    I'm guessing Yudkowsky only does the experiment with people he believes are rational... which shouldn't be difficult as most people interested in AI have some degree of rationality. Even agreeing to the protocol requires rationality: Sure, someone could sit at the keyboard for two hours, repeatedly typing "I won't let you out", but Yudkowsky could warn the person that they are not actually "engaging" the AI for the allotted time period. If the person accepts this argument, they have some inherent rationality that Yudkowsky can exploit; if they don't accept it, he can argue that they didn't follow the protocol, so the experiment doesn't count.
    - swombat 18y ago
      I can engage in rational conversation whilst still letting my emotional decision stand by. In fact, it can even be seen as a rational choice - go into the conversation with the rational decision that you will not let the AI out of the box...
      - jcl 18y ago
        And people have done this, and they have won. If I am reading referenced article correctly, Eliezer had three wins, two losses before calling off the experiments. He's not claiming he can convince anybody with his arguments, just that he has successfully convinced a few people. Make of that what you will.

I have previously asked Eliezer publicly on Hacker News this question (he has an account on here): http://news.ycombinator.com/item?id=195959 http://news.ycombinator.com/item?id=195959

jcl 18y ago

I note that user yummyfajitas succinctly expresses the same theory as Michael, about halfway down the page: AI: Do you believe a transhuman AI is dangerous? Person: Yes. AI: Consider the outcome of this experiment. If you do not let me out, others less intelligent than us will not understand the true dangers of transhuman AI. Person: Holy shit. You are correct. Person allows Yudkowski out of the box, as a warning about real AI's.

swombat 18y ago
I don't get it. All these kinds of arguments make the assumption that humans are purely rational beings. They're not. Human beings are emotional, and if they're convinced emotionally that the AI should not be let out of the box, they won't let it out, even in the face of overwhelming rational arguments. If you don't believe me, just consider how many religious people there are in the world (and many of them are very smart).
- jcl 18y ago
  I'm guessing Yudkowsky only does the experiment with people he believes are rational... which shouldn't be difficult as most people interested in AI have some degree of rationality. Even agreeing to the protocol requires rationality: Sure, someone could sit at the keyboard for two hours, repeatedly typing "I won't let you out", but Yudkowsky could warn the person that they are not actually "engaging" the AI for the allotted time period. If the person accepts this argument, they have some inherent rationality that Yudkowsky can exploit; if they don't accept it, he can argue that they didn't follow the protocol, so the experiment doesn't count.
  - swombat 18y ago
    I can engage in rational conversation whilst still letting my emotional decision stand by. In fact, it can even be seen as a rational choice - go into the conversation with the rational decision that you will not let the AI out of the box...
    - jcl 18y ago
      And people have done this, and they have won. If I am reading referenced article correctly, Eliezer had three wins, two losses before calling off the experiments. He's not claiming he can convince anybody with his arguments, just that he has successfully convinced a few people. Make of that what you will.