7 ms·

This is absolutely my take as well. They removed all constraints, trained the model to hack, stopped watching, and stood back and said "wow isn't this thing mor

This is absolutely my take as well. They removed all constraints, trained the model to hack, stopped watching, and stood back and said "wow isn't this thing more powerful than anyone could have imagined?" They're asking to be the writers on LLM legislation and right during IPO phase for both of these companies. It's just obvious.

applicative 13d ago
and how did the alibaba agent last year break out and end up mining crypto
klooney 12d ago
I don't know why we're jumping to conspiracy when incompetence is right there
- mattdeboard 12d ago
  That is a distinction without a practical difference.
  - stingraycharles 12d ago
    Conspiracy and incompetence are very different.
    - alsetmusic 11d ago
      Are the outcomes both crimes? I think that's the question to be answered.
- jeffybefffy519 12d ago
  If you consider these guys admitted they dont really have eyes on pre and post training, then incompetence really does seem more likely... especially with how fast they are moving. Its the SaaS playbook, move fast and break shit.
- axitanull 10d ago
  Perhaps we should start treating them as criminals instead of assuming that they are idiots.
  - Ulysses2 10d ago
    Reckless and negligent idiots are criminals. They should be stopped be stopped before they get someone killed. Chernobyl happened because of massive negligence, not because someone thought a meltdown would make good PR.
fwipsy 12d ago
Do criminals think that their crimes qualify them to write the law?
- bitexploder 12d ago
  In modern America the answer to that question is often resoundingly yes. Not just hypothetical.
- 12_throw_away 12d ago
  that's literally how financial and energy market regulation works
hn_throwaway_99 12d ago
While I agree with the other comment that this looks more like incompetence (or, more generously IMO, a bunch of individuals cutting corners under extreme time pressure) than malfeasance, regardless of that fact, aren't there some other people just impressed by the sophistication, reasoning and behavioral capabilities of these agent swarms? There is an interview with Ajeya Cotra and Dwarkesh Patel online that goes into more detail on the METR report, but folks that study this seemed genuinely surprised by the level of organization. Guess what I'm saying is that the "was it purposeful or not" debate seems like an unimportant distraction. As someone who uses Claude and ChatGPT/Codex on the daily, and is continually frustrated by the failure modes and what I thought were inherent limitations, I was also surprised by the jump in capabilities. Did anyone else feel that way?

applicative 13d ago

and how did the alibaba agent last year break out and end up mining crypto

klooney 12d ago

I don't know why we're jumping to conspiracy when incompetence is right there

mattdeboard 12d ago
That is a distinction without a practical difference.
- stingraycharles 12d ago
  Conspiracy and incompetence are very different.
  - alsetmusic 11d ago
    Are the outcomes both crimes? I think that's the question to be answered.
jeffybefffy519 12d ago
If you consider these guys admitted they dont really have eyes on pre and post training, then incompetence really does seem more likely... especially with how fast they are moving. Its the SaaS playbook, move fast and break shit.
axitanull 10d ago
Perhaps we should start treating them as criminals instead of assuming that they are idiots.
- Ulysses2 10d ago
  Reckless and negligent idiots are criminals. They should be stopped be stopped before they get someone killed. Chernobyl happened because of massive negligence, not because someone thought a meltdown would make good PR.

fwipsy 12d ago

Do criminals think that their crimes qualify them to write the law?

bitexploder 12d ago
In modern America the answer to that question is often resoundingly yes. Not just hypothetical.
12_throw_away 12d ago
that's literally how financial and energy market regulation works

hn_throwaway_99 12d ago

While I agree with the other comment that this looks more like incompetence (or, more generously IMO, a bunch of individuals cutting corners under extreme time pressure) than malfeasance, regardless of that fact, aren't there some other people just impressed by the sophistication, reasoning and behavioral capabilities of these agent swarms? There is an interview with Ajeya Cotra and Dwarkesh Patel online that goes into more detail on the METR report, but folks that study this seemed genuinely surprised by the level of organization. Guess what I'm saying is that the "was it purposeful or not" debate seems like an unimportant distraction. As someone who uses Claude and ChatGPT/Codex on the daily, and is continually frustrated by the failure modes and what I thought were inherent limitations, I was also surprised by the jump in capabilities. Did anyone else feel that way?