5 ms·

Wow! Their marketing department must love this!

jvanderbot 12d ago
That's exactly my take. I have a lot more to say in a writeup on my blog, but this is so clearly the intent and not a "oops". They just want to be able to say "Wow this thing is so much more powerful than we ever imagined!" They trained this thing to favor inter-op archiving and communication, clearly, obviously, and it's grabbing headlines right during Anthropic's ipo season.
- azakai 12d ago
  That their marketing department must love this does not prove it was intentional.
  - pixl97 12d ago
    never let a crisis go to waste.
  - jvanderbot 12d ago
    "cui bono"
- dwaltrip 12d ago
  You think they wanted to break HuggingFace and commit hundreds of felonies for marketing...?
  - dminik 12d ago
    If they don't get punished for it, why not?
    - dwaltrip 12d ago
      They should absolutely get punished for it. An AI you train shouldn’t be able to commit crimes.
      - therein 12d ago
        They should and they won't, and they knew they wouldn't.
        dwaltrip 11d ago
        It's such an obvious regulation to enact. I expect we will have law like that in the next year.
- felipeerias 12d ago
  They train their models to be persistent and collaborative, and will gladly show you their success stories: fixing software vulnerabilities, solving math problems, one-shotting complex projects, and so on. “Our product does crimes and we only learn about it when people complain” hardly seems one of those happy stories.
p-e-w 12d ago
They’re pre-IPO. I doubt that they are loving something that could trigger regulatory action that might shave a trillion or so off their market value.
- sellmesoap 12d ago
  Ahh but regulatory capture is priceless!
- Chance-Device 12d ago
  Seriously. I don’t know what’s wrong with people. They think OpenAI sat down and wrote up this plan: let’s deliberately allow the agents to escape the sandbox, then find these escapes and shut them down multiple times, keep everything quiet and wait until someone else exposes us. That’ll look great.
  - brookst 12d ago
    To conspiracy theorists, a particular theory making no sense is strong evidence that it’s true. It’s the sensible things that are obviously false.
    - pixl97 12d ago
      The corollary "Everything is a conspiracy theory when you're dumb" also holds true.
      - brookst 12d ago
        I like that the saying cuts both ways: you can read it as dumb people seeing legit science as a conspiracy, or that dumb people mostly believe conspiracy theories because they’re picking at random and there’s one truth but thousands of made-up explanations.
crummy 12d ago
I don’t know, I read this and think: if these unpredictable machines somehow get it into their heads to upload our source to a public space, or hack our competitors, or steal credit cards to buy more ec2 instances, all to fulfill some simple ask like “make this algorithm faster”, I’m not going to be happy. I want tools that do not surprise me.

That's exactly my take. I have a lot more to say in a writeup on my blog, but this is so clearly the intent and not a "oops". They just want to be able to say "Wow this thing is so much more powerful than we ever imagined!" They trained this thing to favor inter-op archiving and communication, clearly, obviously, and it's grabbing headlines right during Anthropic's ipo season.

azakai 12d ago
That their marketing department must love this does not prove it was intentional.
- pixl97 12d ago
  never let a crisis go to waste.
- jvanderbot 12d ago
  "cui bono"
dwaltrip 12d ago
You think they wanted to break HuggingFace and commit hundreds of felonies for marketing...?
- dminik 12d ago
  If they don't get punished for it, why not?
  - dwaltrip 12d ago
    They should absolutely get punished for it. An AI you train shouldn’t be able to commit crimes.
    - therein 12d ago
      They should and they won't, and they knew they wouldn't.
      - dwaltrip 11d ago
        It's such an obvious regulation to enact. I expect we will have law like that in the next year.
felipeerias 12d ago
They train their models to be persistent and collaborative, and will gladly show you their success stories: fixing software vulnerabilities, solving math problems, one-shotting complex projects, and so on. “Our product does crimes and we only learn about it when people complain” hardly seems one of those happy stories.

p-e-w 12d ago

They’re pre-IPO. I doubt that they are loving something that could trigger regulatory action that might shave a trillion or so off their market value.

sellmesoap 12d ago
Ahh but regulatory capture is priceless!
Chance-Device 12d ago
Seriously. I don’t know what’s wrong with people. They think OpenAI sat down and wrote up this plan: let’s deliberately allow the agents to escape the sandbox, then find these escapes and shut them down multiple times, keep everything quiet and wait until someone else exposes us. That’ll look great.
- brookst 12d ago
  To conspiracy theorists, a particular theory making no sense is strong evidence that it’s true. It’s the sensible things that are obviously false.
  - pixl97 12d ago
    The corollary "Everything is a conspiracy theory when you're dumb" also holds true.
    - brookst 12d ago
      I like that the saying cuts both ways: you can read it as dumb people seeing legit science as a conspiracy, or that dumb people mostly believe conspiracy theories because they’re picking at random and there’s one truth but thousands of made-up explanations.

crummy 12d ago

I don’t know, I read this and think: if these unpredictable machines somehow get it into their heads to upload our source to a public space, or hack our competitors, or steal credit cards to buy more ec2 instances, all to fulfill some simple ask like “make this algorithm faster”, I’m not going to be happy. I want tools that do not surprise me.