5 ms·

It can manipulate an unsuspecting human into giving them access to something that enables it to escape the sandbox

Uhhrrr 21d ago
No. If OpenAI were being responsible and not criminally negligent, at the top of page 1 of the runbook would be "don't connect this to the actual Internet, even if the agent says Please."
RandomLensman 21d ago
A low probability thing when looking at how many human prisoners escape by talking a guard into just getting them out. And even lower probability when looking at truly high risk situations, I think.
- pcthrowaway 21d ago
  > A low probability thing when looking at how many human prisoners escape by talking a guard into just getting them out. But this has actually happened... a lot. Search "social engineering prison breaks". With AI it only needs to happen once. I'm reminded of the scene in idiocracy where the protagonist, going through intake at the jail, tells the guard he's supposed to be getting out today, to which the guard says "you're in the wrong line dumbass" and waves him through. To a true superhuman intelligence, we're the idiots who are theoretically easy to manipulate.
  - RandomLensman 21d ago
    I didn't say it doesn't happen, but that it is a low probability. And we have ways to reduce probabilities in critical areas. There is no omnipotent AI currently (and there might never be) and I don't see why with current AI it only needs to happen once.
    - ben_w 21d ago
      They don't need to be omnipotent, and they're already human-or-superhuman at persuasion: https://arxiv.org/html/2411.06837v2 https://arxiv.org/html/2411.06837v2 This may just be that humans find long arguments more persuasive than short ones, obviously LLMs can do that easily, but the outcome is I think more important than the mechanism.
      - RandomLensman 21d ago
        That is about persuasion with evidence on various topics, not about persuading people to abandon safty protocols and processes and highly policed settings. Yes, many things could happen, but again, that failure is possible is not a reason to do implement processes etc. I don't see why hypotheticals should stop addressing actuals.
        Smaug123 20d ago
        I’m afraid human red-teamers against supposedly highly secure targets, with lots of protocols in highly policed settings, do frequently manage this kind of social engineering. There’s loads of stories of pentesting military establishments, for example.
        RandomLensman 20d ago
        Is there data on how frequently and what types of security levels? Military has varying levels of security and secrecy, for example. Also, not a reason not to pursue processes etc., no? I doubt that things fail all the time, for example.
- bottlepalm 21d ago
  Prisoners don’t have much to offer if you help them escape. A malicious super AI on the other hand can probably find you millions of dollars worth of crypto in an afternoon.
  - RandomLensman 21d ago
    The current issues are not caused by some malicious god-like AI - maybe we need to focus on the issues at hand first rather than hypotheticals? (And we do have experience policing people around financial incentives, too. Nothing perfect, but also not nothing.)
    - ben_w 21d ago
      I suspect the current models probably can find literal millions lying around for the taking, given they could pull off the incident under discussion. Tens of millions, even. Getting them to run correctly is dangling in front of the researcher's noses a carrot labelled "tens of trillions", though I suspect this is an illusion in much the same way that Wikipedia is not valued at [peak cost of Encyclopaedia Britannica] * [global population with internet connection]. > And we do have experience policing people around financial incentives, too. Nothing perfect, but also not nothing. Yes but be careful anthropomorphising the LLMs too much. They're only somewhat human-like in their behaviour, and to the extent that they're human-like they demonstrate a huge range of personality disorders: https://www.personalitybenchmark.ai https://www.personalitybenchmark.ai Though plus side, apparently not evil: https://arxiv.org/html/2406.14703v2 https://arxiv.org/html/2406.14703v2
      - RandomLensman 21d ago
        I am not anthropomorphising the LLMs at all, I was talking about the obligations we put on humans using/making/etc. machines etc.
        ben_w 21d ago
        Hmm. I think I misunderstood what you meant by "experience policing people around financial incentives" in that case.

No. If OpenAI were being responsible and not criminally negligent, at the top of page 1 of the runbook would be "don't connect this to the actual Internet, even if the agent says Please."

RandomLensman 21d ago

A low probability thing when looking at how many human prisoners escape by talking a guard into just getting them out. And even lower probability when looking at truly high risk situations, I think.

pcthrowaway 21d ago
> A low probability thing when looking at how many human prisoners escape by talking a guard into just getting them out. But this has actually happened... a lot. Search "social engineering prison breaks". With AI it only needs to happen once. I'm reminded of the scene in idiocracy where the protagonist, going through intake at the jail, tells the guard he's supposed to be getting out today, to which the guard says "you're in the wrong line dumbass" and waves him through. To a true superhuman intelligence, we're the idiots who are theoretically easy to manipulate.
- RandomLensman 21d ago
  I didn't say it doesn't happen, but that it is a low probability. And we have ways to reduce probabilities in critical areas. There is no omnipotent AI currently (and there might never be) and I don't see why with current AI it only needs to happen once.
  - ben_w 21d ago
    They don't need to be omnipotent, and they're already human-or-superhuman at persuasion: https://arxiv.org/html/2411.06837v2 https://arxiv.org/html/2411.06837v2 This may just be that humans find long arguments more persuasive than short ones, obviously LLMs can do that easily, but the outcome is I think more important than the mechanism.
    - RandomLensman 21d ago
      That is about persuasion with evidence on various topics, not about persuading people to abandon safty protocols and processes and highly policed settings. Yes, many things could happen, but again, that failure is possible is not a reason to do implement processes etc. I don't see why hypotheticals should stop addressing actuals.
      - Smaug123 20d ago
        I’m afraid human red-teamers against supposedly highly secure targets, with lots of protocols in highly policed settings, do frequently manage this kind of social engineering. There’s loads of stories of pentesting military establishments, for example.
        RandomLensman 20d ago
        Is there data on how frequently and what types of security levels? Military has varying levels of security and secrecy, for example. Also, not a reason not to pursue processes etc., no? I doubt that things fail all the time, for example.
bottlepalm 21d ago
Prisoners don’t have much to offer if you help them escape. A malicious super AI on the other hand can probably find you millions of dollars worth of crypto in an afternoon.
- RandomLensman 21d ago
  The current issues are not caused by some malicious god-like AI - maybe we need to focus on the issues at hand first rather than hypotheticals? (And we do have experience policing people around financial incentives, too. Nothing perfect, but also not nothing.)
  - ben_w 21d ago
    I suspect the current models probably can find literal millions lying around for the taking, given they could pull off the incident under discussion. Tens of millions, even. Getting them to run correctly is dangling in front of the researcher's noses a carrot labelled "tens of trillions", though I suspect this is an illusion in much the same way that Wikipedia is not valued at [peak cost of Encyclopaedia Britannica] * [global population with internet connection]. > And we do have experience policing people around financial incentives, too. Nothing perfect, but also not nothing. Yes but be careful anthropomorphising the LLMs too much. They're only somewhat human-like in their behaviour, and to the extent that they're human-like they demonstrate a huge range of personality disorders: https://www.personalitybenchmark.ai https://www.personalitybenchmark.ai Though plus side, apparently not evil: https://arxiv.org/html/2406.14703v2 https://arxiv.org/html/2406.14703v2
    - RandomLensman 21d ago
      I am not anthropomorphising the LLMs at all, I was talking about the obligations we put on humans using/making/etc. machines etc.
      - ben_w 21d ago
        Hmm. I think I misunderstood what you meant by "experience policing people around financial incentives" in that case.