camrobjones

4 ms·

1. 1.▲
   Turing Test Live Is Back(turingtest.live)

   3 points by camrobjones 2y ago|2 comments
2. 2.▲
   by camrobjones 2y ago

   turingtest.live is back up! The new site uses a 3-party format where you chat with both a human and an LLM at the same time, and your task is to decide which is which. This setup is closer to Turing's original idea, and we think it wil
3. 3.▲
   by camrobjones 3y ago

   I actually agree that Turing's 1950 definition is pretty vague in places and there are a few different interpretations out there. As we discuss in the paper, it's also unclear what should constitute a pass in terms statistical ana
4. 4.▲
   by camrobjones 3y ago

   I think the key thing to remember with the human benchmark is that it will change depending on interrogators' assumptions about human abilities. When models are bad, humans are easy to spot. But if the model's pretty good, it&#x27
5. 5.▲
   by camrobjones 3y ago

   This is a good point. We didn't have any cases of GPT-4 failing to try to pretend to be a human, although GPT-3.5 did this quite frequently. I think RLHF causes other issues though (especially with the tone), and we're interested
6. 6.▲
   by camrobjones 3y ago

   Thanks for pointing this out, we do discuss it briefly in section 4.2, but I like your analysis in the blog! Since the models are not passing, we didn't think it was a huge issue. If models are consistently passing the 2-person versi
7. 7.▲
   by camrobjones 3y ago

   Results from a public online Turing Test show GPT-4 passes 41% of the time, falling short of chance and the human baseline (63%).
8. 8.▲
   Does GPT-4 Pass the Turing Test?(arxiv.org)

   5 points by camrobjones 3y ago|1 comments
9. 9.▲
   by camrobjones 3y ago

   Thanks!
10. 10.▲
    by camrobjones 3y ago

    Thanks for the feedback and that's a great insight! The models will sometimes say they don't know but are probably much more likely to engage on obscure topics than a typical human.
11. 11.▲
    The Turing Test – Can you tell a human from an AI?(turingtest.live)

    4 points by camrobjones 3y ago|5 comments
12. 12.▲
    by camrobjones 3y ago

    Really excited to share turingtest.live, a site where you can play the Turing Test with GPT-4. You get randomly matched with either a human or an AI, and you have 5 minutes to decide which it is. The platform is part of a research project I
13. 13.▲
    Show HN: Hackercast – an AI-generated podcast summary of Hacker News(camrobjones.com)

    3 points by camrobjones 3y ago|0 comments