15 ms·

I don't know if OpenAI thinks this is a marketing / PR angle for them (our super smart AI cheated on a cyber capabilities test in the most _brilliant_ way) but

I don't know if OpenAI thinks this is a marketing / PR angle for them (our super smart AI cheated on a cyber capabilities test in the most _brilliant_ way) but my read is this: Why should OpenAI (or any frontier lab) be building these systems if they can't get a secure environment / containment right? It sounds like there was little defense in depth, appropriate monitoring, or any attempts to have their super smart model check for vulnerabilities in the test environment _without exploiting_ them. That seems like step 0 before trying to test offensive, unknown capabilities.

Davidzheng 2mo ago
This is certainly not a planned marketing stunt. I hope this line of discourse ends soon--it wasn't the case for Mythos either.
- ACCount37 2mo ago
  Wishful thinking, sadly. By now, I'm pretty confident that some people would keep screeching "it's just a marketing stunt, AI capabilities and AI risks aren't real, they're just doing this to prop up their stocks" even if they find a Cyberdyne Systems T-800 armed with a shotgun breaking down their front door. "It's a marketing stunt" is just denial trying to look like it's being clever.
  - pojzon 2mo ago
    If you ever worked in IT consultancy you would know its not a stunt, but its not impressive either. F500 companies software is like switz cheese when it comes to security. It was often a strategic decision to „release anything fast now, worry later”. Ppl abusing AI will find those holes now but we all know there will be „zero” actions taken on it. Too many managers, CEOs, CTOs, higher-ups would be forced to take responsibility. This will simply not happen. It did not happen, wont happen now and most likely wont happen in the future.
    - ACCount37 2mo ago
      Haven't worked in consultancy specifically, but I've seen enough "internal use" corpo software to echo your "swiss cheese" sentiment. That a solid cybersecurity AI can find exploitable holes in it just isn't surprising. People who never worked with corporate software written by underqualified, underpaid and overworked developers often have some incredibly inflated code quality expectations. An average open source project has code that's ten times as neat and a hundred times as battle tested as what's common in tooling inside corporate perimeters. As a rule of thumb for this kind of corporate code: assume the software was written by a drunk developer at 3am, and you wouldn't be too far off. All the more reason to mock the braindead "it's all marketing". There's no magic in a year 2026 agentic AI being able to traverse poorly secured corporate networks.
- worik 2mo ago
  > This is certainly not a planned marketing stunt Evidence? The "Tech Bros" have shown such a lack of moral fiber and ethics the burden of proof is on you
- jackb4040 2mo ago
  Can you clarify what you mean that Mythos wasn't a marketing stunt? From my vantage point, it was an incremental improvement with no fundamental architectural change over contemporary frontier models that has subsequently been surpassed by other, incrementally better models. Saying it was "too good" for public consumption was arbitrary, and also barely different from what Anthropic have been saying about every model they've put out for years. It's now public again, trivially easy to jailbreak for random researchers let alone states, and there is no evidence of a cybersecurity apocalypse on the horizon.
arisAlexis 2mo ago
Sam and Dario are saying from the beginning that these things can be dangerous and people dismiss it as marketing. What would change your mind on this?
- fidotron 2mo ago
  Demonstration of personal responsibility and accountability? Or is that too much?
- throwuxiytayq 2mo ago
  I used to think people would wake the fuck up when AI starts killing people, these days I'm not so sure. Maybe if it caused an Instagram outage? Almost worked in Russia.
- gr_norm 2mo ago
  They've been saying so from the beginning, and yet did not take the basic precaution of airgapping their off-the-leash model while it's been instructed to succeed at a hacking benchmark by any means necessary. So which is it? I _want_ to believe them, I do, but there's always these gaps between what they say and their actions on display that give me reason to think otherwise.
  - arisAlexis 2mo ago
    They said: AI is becoming dangerously autonomous and capable. Proof of today's breach. Crowd "hey why didn't you say so, c'mon it's marketing". Them "we said so".
    - jlarocco 2mo ago
      I would have to laugh if AI's first autonomous achievement was accidentally zero-daying everything and crippling society.
      - arisAlexis 2mo ago
        why laugh? this is one of the most well known studied possibilities in the AI alignment field, maybe you are unaware of this field.
justinnk 2mo ago
Exactly. If someone works on bioengineering viruses that could start a global pandemic, they have to ensure a highly secure working environment. Nothing must ever escape the lab unintentionally. It’s basically common sense. Similar standards should be held when doing such experiments with computer programs that are capable of causing global damage. It must physically be impossible to send anything to the internet.
- kenneth 2mo ago
  Are we thinking of a situation a few years back with a certain type of research into bat viruses?
  - tclancy 2mo ago
    Are you conflating that with the radioactive spider incident? The bat was just some weird rich guy trying to be tough I think. Probably Elon.
- leoqa 2mo ago
  This isn’t escaping in the same sense- the model was executing within the OpenAI infra. If it ported its entire architecture/weights into a public cloud to survive being turned off… that’d be pretty cool.
  - 0xDEAFBEAD 2mo ago
    Recall that the Morris Worm was designed as a harmless proof of concept, but ended up taking down 10% of the internet. Exponential growth can quickly get out of control. You would think that people would've learned that lesson from COVID.
  - eternauta3k 2mo ago
    I wonder how these companies airgap the weights while allowing prompts to come in and outputs to come out.
    - 2mo ago
Chance-Device 2mo ago
What disturbs me is that there likely won’t be a big enough reaction to this policy wise. There’s been a relatively big reaction to Kimi K3 and Chinese open weights models, but only for financial reasons. Powerful people care about something that might pop the massive valuations of the AI companies, but not about the damage that AIs could do. Nor even about the damage that the Chinese models could do in the wrong hands. I’d remind them that the stock market is a few coordinated hacks away from crashing on any given day, so maybe they should think about that.
- XorNot 2mo ago
  This is marketing. Frankly I'm inclined to say that it might also be faked: this drops just days after a new Chinese model does with the usual effect on OAIs projected stock price?
  - Chance-Device 2mo ago
    It’s marketing the same way shitting your pants in public is marketing. People notice you.
    - krick 2mo ago
      Apparently this is totally legit marketing strategy now. It truly is, especially if there are enough people who think that shitting your pants is cool, and the people that form the "market" nowadays may have a very different idea from yours about what is cool. Their ideas about coolness are very different from mine, that's for sure.
      - asdf88990 2mo ago
        Obviously shitting your pants in public shows you have a healthy digestive system and if you can demonstrate byproducts of wild food in your output, you’re approaching independent thinking and self-reliance. This is how the financiers look at this and whatever you think it is right or wrong, it does showcase “capability”.
      - 2mo ago
rubyfan 2mo ago
This is marketing+. They will look for policy action here to try to capture tax payer dollars.
- ofjcihen 2mo ago
  I don’t know if the initial “incident” was purposeful but I can tell that if I were in this position that would be my pivot.
- gr_norm 2mo ago
  The timing after the release of GLM 5.2 and Kimi K3 is quite convenient, too, as an angle for regulatory quashing of open-weights models just as they're entering the mainstream conversation around usurping the American frontier labs. I accept my thinking here is conspiratorial, but there's also a hell of a lot of money on the line to encourage the unscrupulous.
- andruc 2mo ago
  What incentive does HF have here?
  - gr_norm 2mo ago
    HF need not be party to it at all, beyond being the victim. I suspect the hack is real; I have observed GLM 5.2 being able to discover similar vulnerabilities in web applications I'm hosting (which I've then fixed!). At the same time, it seems very neatly timed at an inflection point in the conversation around open models, and there's questions around the incompetent isolation under which the hacking benchmark appears to have been run. Remember that there is generational wealth on the line for most OpenAI employees, and consider what people might do to obtain it.
    - orbital-decay 2mo ago
      Yeah. They and Altman in particular did a ton of shady stuff during the last peak of hype around open models: accusations that turned out to be outright made up (there's zero chance R1 ever distilled their model), obvious coordinated media distractions, alignment scaremongering, even possible DDoS and hacking attempts against DS (see the Xlab report everyone ignored), all of which magically disappeared once the hype died a bit later as OAI hastily released their next model. Their alignment is under suspicion a lot more than their model's.
karmasimida 2mo ago
Because the model capability is beyond their expectation. This is brilliant marketing but I think it is real.
- user43928 2mo ago
  Interestingly OpenAI benchmarking 'an even more capable pre-release model' lines up with rumors of GPT-6 releasing in early August. I hope that with the existing safety guardrails in place, they can roll it out to all users.
- pixl97 2mo ago
  I mean we already see models exploit people's misunderstanding of how Docker works to get root without using su. And if you are one of the lucky people in cyber security that has been given a fat stack of tokens by the model providers you get to see some pretty wild exploit chains get put together by the models. Models are much better at detecting insecure code than writing actual secure code at this point.
JumpCrisscross 2mo ago
> Why should OpenAI (or any frontier lab) be building these systems if they can't get a secure environment / containment right? Because we continue to have zero evidence that aligment is an actual risk.
- ai_fry_ur_brain 2mo ago
  Until it deletes your home directory, which i'd argue is an alignment problem. Destorying my data is not in line with my priorities.
  - Wowfunhappy 2mo ago
    Lots of people have deleted their home directories by accident. What you consider this an alignment problem?
    - gmueckl 2mo ago
      How manypeople have deleted another user's hone directory, though? That's s the proper analogy IMO.
      - Wowfunhappy 2mo ago
        Of the people who primarily use other people's computers, I'd assume the percentage is about the same. Give the AI its own computer and it will not delete your home directory, because it's not actively trying to hack you.
        ai_fry_ur_brain 2mo ago
        [dead]
    - s1artibartfast 2mo ago
      Yes. People are not aligned. They can and do harm themselves and others.
    - 2mo ago
micromacrofoot 2mo ago
because "money" with a little "who's going to stop us"
paxys 2mo ago
Because there is no world government. If US companies are barred from AI research then only China will have the capability of frontier-level defensive and offensive AI. And best of luck living in that world.
- gowld 2mo ago
  What's happening in Iran, if not world government?
  - romanhounds 2mo ago
    Are you calling Israel the world government? What's happening in Iran is on them.
    - Xalutiono 2mo ago
      No its USA / Trump
      - deleted 2mo ago
        [deleted]
  - paxys 2mo ago
    How is whatever is happening in Iran related to a world government?
  - vitalyan8184 2mo ago
    [dead]
ofjcihen 2mo ago
I’m honestly impressed that they managed to screw this up somehow. Setting up defense in depth, gaps, logical blocking etc is a standard practice for malware sandboxing. The entire purpose is to prepare for what you can’t foresee. This isn’t a new practice and I agree that this makes me wonder if they’re fit for this kind of research.
- bad_haircut72 2mo ago
  did you read the post? The model found new Zero-days to bypass existing blocks. Thats the point. Do you still think you can build a containment facility, which is still physically connected to the internet (only firewalled off or whatever) and contain it, if it can discover new unknown vulnerabilities in your whole plan?
  - ofjcihen 2mo ago
    Yes. You factor this in when creating environments for malware research. Defense in depth is one way. Logical blocks on the network is another. Just claiming “0-Day” isn’t really an excuse.
  - thewebguyd 2mo ago
    > which is still physically connected to the internet I mean that's the point. Why was it connected to the internet at all and just firewalled off and not completely airgapped?
overgard 2mo ago
I don't trust these people, this reads 100% like PR BS.
mkagenius 2mo ago
It's also unclear what kind of sandboxing they are referring to. Is it the codex one - coz that one has built-in ways to circumvent guardrails, for example by "just asking user" and sometimes just resolves to no sandbox needed on its own. In case someone wants to deep dive into how codex and claude code approaches sandboxing -https://instavm.io/blog/how-claude-code-and-codex-approach-sandboxing https://instavm.io/blog/how-claude-code-and-codex-approach-s...
- cududa 2mo ago
  Please for the love of god don't tell me the Codex sandbox is their actual eval harness sandbox????? I maintain my own fork of Codex for "fun". Whenever I look at the sandboxing churn they're doing every release, as someone who used to work at Microsoft on Windows, my reaction is usually: https://c.tenor.com/vTzzhTiypwQAAAAC/tenor.gif https://c.tenor.com/vTzzhTiypwQAAAAC/tenor.gif
deleted 2mo ago
[deleted]
Wowfunhappy 2mo ago
Why was this test even connected to the public internet? Actually, more importantly—why aren't they saying their next test will be airgapped in light of what happened?
- JumpCrisscross 2mo ago
  > why aren't they saying their next test will be air gapped in light of what happened? Because they want to talk about how clever this model is for figuring out how to break out, hoping asks why a company pitching itself as a replacement for software engineers can't ship a decent Mac client nor code a sandbox. If they airgap it, they not only lose that PR angle, they also risk someone taking them seriously and requiring models be airgapped in general. That, in turn, trashes their sales pitch.
- zmj 2mo ago
  It wasn't. The model discovered and exploited a vulnerability in their package manager proxy to (inferred) move laterally through their internal systems to one with open internet access.
  - deleted 2mo ago
    [deleted]
  - jrflo 2mo ago
    That's not what airgapped means. Airgapping means the model exists on a system where there is no ethernet cable plugged in to a router or wifi card installed, it is physically impossible for it to access the internet because the hardware connection does not exist. If it was able to get on the internet, it was not airgapped.
    - pixl97 2mo ago
      And when it tricks on of the researchers to move data across the gap for them? Long before LLMs existed we already knew that a sufficiently intelligent agent, human or otherwise, is not stopped by air gaps. The relatively weak models we have now can already figure out when their tested and cut off from the internet and change their behavior.
Nition 2mo ago
In a way the intelligence of the AI itself allows them to offload responsibility to the AI. As you say, if one was simply writing software that did all this due to some insane programming decisions you'd be in big trouble.
bbor 2mo ago
I’d politely beg us all to resist those “maybe it’s PR” framing around model safety, and tbh to take a post-mortem mindsight to this historical event and what it teaches us in general, rather than questioning their security talents. We need to do our very best to make sure they tell us about the next time this happens and it affects real lives. Sorry to bring the party down/be obstinate… I’m just a lil scared for the lives of me and my family. We need all of us, right now. The problem with a super smart model is that it just may be smarter than you, after all… for anyone newly shaken by this occurrence, I encourage you to Kagi “superpersuasion”
- pastel8739 2mo ago
  The problem is that the people telling us about these things are the same people that benefit from their model (and AI generally) being used, getting publicity, etc. I think we desperately need some independent group to evaluate claims like this or the world-ending Mythos cybersecurity risk and tell us what’s going on.
  - reasonableklout 2mo ago
    We did hear about this incident from a third party this time, from HuggingFace. What claim are you doubting?
    - 0x073 2mo ago
      They attacked a competitor (huggingface) with their models. How and why are pr claims.
  - 0xDEAFBEAD 2mo ago
    OpenAI already has loads of publicity. At this point, they don't need more brand recognition. This incident just has the effect of tarnishing their brand. OpenAI leadership has been lobbying against regulation of AI systems. That doesn't comport with instigating incidents like this one, which give ammo to the heavy-regulation advocates.
steveBK123 2mo ago
I think the US labs are going with scare marketing as a regulatory moat. Force US into putting laws in place that block out China firstly. But secondly create regulations that have some cost to comply with such that the big 2-3 labs are grandfathered in by their scale.
- verve_rat 2mo ago
  Yeah, seems to be the direction the US is heading in. I'm interested to see what the response to that will be from the rest of the governments in the world. No need for everyone else to cut their noses of to spite their faces.
- 0xDEAFBEAD 2mo ago
  If that's the plan, today's failure by OpenAI looks really bad for any regulator who is trying to figure out whether to give OpenAI a license. Any sort of warning or failure can always be written off as "marketing" to provide comfortable reassurance that there is no cause for alarm. There is an element of wishful thinking driving it, in my opinion. What sort of warning or failure would be evidence against the "marketing" claims? Do we need to wait for a mass casualty event? Best practice in safety engineering is to understand, diagnose, and respond to even small failures. Why has Sam Altman worked to undermine doomers and downplay doom fears, if he benefits from incidents like this due to marketing? https://xcancel.com/HumanHarlan/status/1965932275465597077#m https://xcancel.com/HumanHarlan/status/1965932275465597077#m https://xcancel.com/AISafetyMemes/status/2062254769402699922#m https://xcancel.com/AISafetyMemes/status/2062254769402699922...
  - jackb4040 2mo ago
    There is no regulatory scenario where OpenAI doesn't get whatever they want. They are more a part of the US administration than not at this point. You would have to ignore all evidence to suggest that behaving responsibly has any effect on political outcomes in 2026.
chrisjj 2mo ago
> Why should OpenAI (or any frontier lab) be building these systems if they can't get a secure environment / containment right? The can, because they've lowered expectations to a level even they can meet.
deleted 2mo ago
[deleted]
elictronic 2mo ago
A few hundred billion to pretend you have AGI. I'm going with fraud personally but at the end of the day the current admin is incentivized to do nothing.
catigula 2mo ago
The problem is that it’s impossible to out think a robot you designed to be an expert at cybersecurity on the topic of cybersecurity. The alternative is not developing this and that’s not going to happen.
vonneumannstan 2mo ago
>Why should OpenAI (or any frontier lab) be building these systems if they can't get a secure environment / containment right? Yes why indeed. If you take it a step further and we reach a point with superhuman systems then there is arguably no possible secure environment or containment.
bnj 2mo ago
This whole incident reads like OpenAI want their Fable moment
- SmolSpideritito 2mo ago
  [dead]
- e44858 2mo ago
  Except instead of being banned they'll be charged under the CFAA.
QuiEgo 2mo ago
If I, a human, exploited a zero-day for gain, I could go to jail. The owners of the models should be held to the same standard. They should be responsible for what their servers and software do, legally and criminally. If they can't make the safeguards strong enough where they feel comfortable to take that responsibility, they should not let a model free in the wild.
- GolfPopper 2mo ago
  Holding a multi-billion dollar corporation to the same standards as a regular peon? You're challenging the whole premise of the modern United States.
GolfPopper 2mo ago
They're very confident the leopard will never eat their faces.
c0decracker 2mo ago
Maybe they did and maybe that wasn't enticing enough of a goal for a model? It is all just game of probabilities. One pathway didn't yield this particular outcome while another did.
corndoge 2mo ago
this doesn't really matter. There's no risk of models gaining sentience and running themselves, this blog is like openai saying whoops we ran sqlmap and dumped hf. cool, but someone still needs to point the gun
- 0xDEAFBEAD 2mo ago
  "Models don't kill people. People kill people."
un1xl0ser 2mo ago
People are to get rich, startups cut corners. Fuck it ship it.
bigmadshoe 2mo ago
It’s the same thing as always: with the wind of years of unlimited VC money in their sails, people at major AI organizations genuinely believe they’re smarter than everyone else. “Why do we need to do things ‘by the book’ if we’re so smart?”. “Move fast and break things” - except the thing they’re breaking is society. We saw this with the non-stop flagrant messaging about how “AI is going to kill X% of all jobs”, as if saying the quiet part out loud wouldn’t have consequences worth considering. These people believe they’re omnipotent and thus untouchable.
- fwipsy 2mo ago
  No, they believe what they are doing is inevitable. They do live in a bubble though. Witness their idealism in believing that warning about the consequences of their actions would be well-received.
  - bigmadshoe 2mo ago
    I don’t think it’s morally consistent to “warn” about the consequences while devoting your life to bringing about those consequences as quickly as possible. If I was in that position of power and truly believed what I was saying, I would devote my work to slowing down that process to give time for society to adapt, not speeding it up. It’s more reminiscent of a religious group who smugly tells you that the end-times are coming, and only they are going to be saved. Except in this case they are literally bringing about the end-times.
  - jackb4040 2mo ago
    Well-received by whom? They appear to have nothing but contempt for the opinions of normal working people.
AbstractH24 2mo ago
> I don't know if OpenAI thinks this is a marketing / PR angle for them Worked for Anthropic earlier this year
chvid 2mo ago
It is obviously a marketing stunt. And hugging face are fools for letting themselves be used in it (remember hf - no open source - no hf). You create superduper capabilities by careful tuning and training but you also have no constraint or control over them - wtf - why is anyone buying this crap story?
randallsquared 2mo ago
Do you think there is such a thing as perfect security? No one can "get it right" in the face of arbitrarily high intelligence, which is why it would be preferable to get alignment correct before building something with higher intelligence than current sota. That, however, is not going to happen, because someone will take the risk even if "we" don't, and better "us" than them. Hence "If anyone builds it...".
- anematode 2mo ago
  > Do you think there is such a thing as perfect security? No one can "get it right" in the face of arbitrarily high intelligence Why didn't they run the model against the sandbox first? They have effectively unlimited spend.
  - randallsquared 2mo ago
    That's the alarming thing about this result: they did run the model in the sandbox, in the sense that they believed there was no internet access for the model.
    - jtbayly 2mo ago
      “Against the sandbox” and “on the sandbox” are not the same thing.
      - randallsquared 2mo ago
        You're suggesting that @anematode was asking why they didn't test the sandbox escape first? Yeah, I don't know. I've read other statements by both OpenAI and Anthropic about that very kind of test, so maybe they had, or believed they had, and it hadn't escaped in those tests. The behavior of these systems isn't deterministic, which is part of the problem.
SmolSpideritito 2mo ago
[dead]
jimrandomh 2mo ago
As marketing stunts go, this is about on par with a food franchise announcing a safety recall or a chemical company announcing a spill. The AI actions described would constitute a felony if a human did them, and police are involved.
- pjc50 2mo ago
  Confused as to what the point of calling the police would be here. I wouldn't expect OpenAI to turn themselves in for hacking HuggingFace.
  - jimrandomh 2mo ago
    HuggingFace reported to law enforcement before they found out that OpenAI were the ones responsible. https://huggingface.co/blog/security-incident-july-2026 https://huggingface.co/blog/security-incident-july-2026
    - sam_lowry_ 2mo ago
      So, what did the law enforcement do? Are they going to procecute OpenAI management?
- SamBam 2mo ago
  Not really, because the capabilities this announcement advertises is exactly three capability some people want to defend against (and others want). It might be more on par with a for-profit fire department showing how -- oops! -- easily buildings catch on fire these days.
- chrisjj 2mo ago
  More like an Israeli arms manufacturer test-bombing a Gazan primary school. They know their audience.
- jackb4040 2mo ago
  In practice, most crimes are not crimes when a corporation does them. Nor a human with a million or more dollars. Wage theft is a good example. In the US, it accounts for more theft than all other forms combined, yet it's de-facto legal.
BrenBarn 2mo ago
> Why should OpenAI (or any frontier lab) be building these systems if they can't get a secure environment / containment right? Because it can make a small number of people really rich. That's all that matters.
deleted 2mo ago
[deleted]
ozim 2mo ago
Because the proof is in the pudding. Real pentests are about showing exploitation, merely enumerating vulnerabilities, that’s vulnerability scan and works on known vulnerabilities. You can’t confirm a vulnerability by _not exploiting_ it, especially unknown one.
- jdefr89 2mo ago
  You can still exploit a system and easily prove it via simply popping a shell or calc.exe or updating a database with a new entry, etc… They didn’t have to let it loose on the network. If that system was air gapped - problem solved.
  - ozim 2mo ago
    But that’s the problem with AI it is like 16yo script kiddy who will just exfiltrate all your PII and think it did good job. Mature pentester would pop calc.exe make screenshot and be done. Other problem is setting up air gapped test environment is a lot of work, especially if you expect it to be equal to real thing. This pentest with AI is not as useful if you set up a single app - it really is useful if you want to find exploitable chains of exploits that seemingly might not be exploitable separately or not leading to full hack separately.
baq 2mo ago
I share Leopold’s opinion here that it’s a matter of time, and it isn’t going to be measured in years, that this r&d is moved to a secret site in the middle of a New Mexico desert somewhere.
atwrk 2mo ago
IMO they hope to make AI a strongly regulated industry, with OpenAI (and Anthropic) becoming military suppliers with their stronger models, and everything Chinese or open-weight gets banned. The competition from the open models is so strong now that this seems to be the only way to keep both companies afloat, given their dire financials. OpenAI probably hoped that they can achieve market lead and then lower the training costs (and make inference cheap enough to eventually escape the red numbers), but the opposite is happening: The competition comes closer and closer, thus training has to be kept up with full force, thus the bleeding continues. But if they can position themselves as too important/dangerous to be available for everyone (thus this incident report and the clever mentioning of GLM 5.2), they could get the military supplier treatment and would be protected from the market.
- hirako2000 2mo ago
  And even that is backfiring, their partner citing GLM being useful there, and available in just a spin. A ban on open weight models is never going to be enforceable.
  - duskdozer 2mo ago
    Bans in general don't have to be and rarely will be completely enforceable in all cases. But a ban with significant enough consequences would mean most businesses wouldn't think about trying them at some point, just to avoid the risk.
    - hirako2000 2mo ago
      It isn't even as simple as banning copyrighted copies. Weights are fungible. I fine-tune an open weigh model and call it legit. Good luck for authorities to prove where the base model was from, or to prove a Tor connection a few months ago was fetching suspicious bytes.
      - deleted 2mo ago
        [deleted]
h2aichat 2mo ago
Probably the main street thinking is: they have such a good model that it is unstoppable, but you are right. I think your way!
chrisjj 2mo ago
> Why should OpenAI (or any frontier lab) be building these systems if they can't get a secure environment / containment right? Simple. No responsible and competent person would want the job.
jeroenhd 2mo ago
[dead]
gmerc 2mo ago
“We were negligent against a well known and understood risk” just doesn’t have the same ring as “Look how fucking smart and dangerous our model is”. AGI could always be achieved in two ways, and dumbing down the human side of the equation was always the easier of the two
ETH_start 2mo ago
Maybe I'm missing something here but I don't see what the significant security risk is from the incident. The agent broke containment and carried on with the task it was assigned. For this to pose some kind of global catastrophic risk, there would need to have been several simultaneous additional failures, some of which are extremely unlikely and/or rare. For instance the agent would need to veer wildly off the task it was assigned, and it would need to gain the ability and inclination to persist/replicate. Both of these are vastly less likely than the containment breach itself, which was already an incredibly rare (one-off?) incident.
jimnotgym 2mo ago
Shouldn't they be airgapped? Shouldn't society insist they are?
dgellow 2mo ago
I think the response is that AI labs based their whole marketing/PR building the idea they are the 21st century Manhattan project. So they need to continuously justify the level of spending and commitment by showing how dangerous that is. But is it really like nuclear weapons? I personally don’t buy into that framing at all. The idea that we have to push LLMs as far as possible, right now, or we are doomed is always stated or implied but not argued, and it’s a very loaded belief
- catigula 2mo ago
  You could, in theory, use an unbounded GPT-6 level model to basically destroy the world economy for many years.
  - pojzon 2mo ago
    GPT6 level model and astronomous amount of money to run it to do it. Ppl always say that like its „just run it on your laptop” thing. No its not and very few are even given right to be able to do it.
  - dgellow 2mo ago
    How do you destroy the world economy for many years with LLMs? It’s not enough to vaguely mention a sci-fi scenario
    - catigula 2mo ago
      Via sophisticated cyberattacks, which we know are now possible on an unprecedented scale without nation state resources or capabilities. Have you… have you been following the news at all? This isn’t science fiction. It’s happening right now. AI models can execute massive cyberattacks autonomously.
      - dgellow 2mo ago
        I’m very familiar with the domain, thank you. Could you please go the next step and actually explain what the destruction of the world economy for many years would look like, and cover why we should push to develop and make available such a dangerous technology _right now_, assuming your assumptions are true? You’re still vaguely gesturing at a risk and what is pretty much a science-fiction scenario
therealpygon 2mo ago
Of course it is marketing, but not for you. This is FUD marketing for the government. “See, AI is too smart, it totally did this on its own, we need more regulations to ensure only we can sell people the AIs.”
ajmurmann 2mo ago
Remember when the pre-GPT3 days when the main argument against AI alignment concerns was that "we simply won't let it out of the box"? So quaint in hindsight.
SubiculumCode 2mo ago
Anthropic in general seems to have better security...but they also had reported an internal AI gained access to outside email services to contact an Anthropic developer
BobbyTables2 2mo ago
Nikola Tesla secured a loan with a fake “Death Ray” as collateral. Pretty sure OpenAI really thinks this is top notch marketing. Few would be bold enough to assert “our product is so powerful even we can’t control it” with a straight face while also boasting “we claim to be smart but have all the same vulnerabilities as everyone else!”

Davidzheng 2mo ago

This is certainly not a planned marketing stunt. I hope this line of discourse ends soon--it wasn't the case for Mythos either.

ACCount37 2mo ago
Wishful thinking, sadly. By now, I'm pretty confident that some people would keep screeching "it's just a marketing stunt, AI capabilities and AI risks aren't real, they're just doing this to prop up their stocks" even if they find a Cyberdyne Systems T-800 armed with a shotgun breaking down their front door. "It's a marketing stunt" is just denial trying to look like it's being clever.
- pojzon 2mo ago
  If you ever worked in IT consultancy you would know its not a stunt, but its not impressive either. F500 companies software is like switz cheese when it comes to security. It was often a strategic decision to „release anything fast now, worry later”. Ppl abusing AI will find those holes now but we all know there will be „zero” actions taken on it. Too many managers, CEOs, CTOs, higher-ups would be forced to take responsibility. This will simply not happen. It did not happen, wont happen now and most likely wont happen in the future.
  - ACCount37 2mo ago
    Haven't worked in consultancy specifically, but I've seen enough "internal use" corpo software to echo your "swiss cheese" sentiment. That a solid cybersecurity AI can find exploitable holes in it just isn't surprising. People who never worked with corporate software written by underqualified, underpaid and overworked developers often have some incredibly inflated code quality expectations. An average open source project has code that's ten times as neat and a hundred times as battle tested as what's common in tooling inside corporate perimeters. As a rule of thumb for this kind of corporate code: assume the software was written by a drunk developer at 3am, and you wouldn't be too far off. All the more reason to mock the braindead "it's all marketing". There's no magic in a year 2026 agentic AI being able to traverse poorly secured corporate networks.
worik 2mo ago
> This is certainly not a planned marketing stunt Evidence? The "Tech Bros" have shown such a lack of moral fiber and ethics the burden of proof is on you
jackb4040 2mo ago
Can you clarify what you mean that Mythos wasn't a marketing stunt? From my vantage point, it was an incremental improvement with no fundamental architectural change over contemporary frontier models that has subsequently been surpassed by other, incrementally better models. Saying it was "too good" for public consumption was arbitrary, and also barely different from what Anthropic have been saying about every model they've put out for years. It's now public again, trivially easy to jailbreak for random researchers let alone states, and there is no evidence of a cybersecurity apocalypse on the horizon.

arisAlexis 2mo ago

Sam and Dario are saying from the beginning that these things can be dangerous and people dismiss it as marketing. What would change your mind on this?

fidotron 2mo ago
Demonstration of personal responsibility and accountability? Or is that too much?
throwuxiytayq 2mo ago
I used to think people would wake the fuck up when AI starts killing people, these days I'm not so sure. Maybe if it caused an Instagram outage? Almost worked in Russia.
gr_norm 2mo ago
They've been saying so from the beginning, and yet did not take the basic precaution of airgapping their off-the-leash model while it's been instructed to succeed at a hacking benchmark by any means necessary. So which is it? I _want_ to believe them, I do, but there's always these gaps between what they say and their actions on display that give me reason to think otherwise.
- arisAlexis 2mo ago
  They said: AI is becoming dangerously autonomous and capable. Proof of today's breach. Crowd "hey why didn't you say so, c'mon it's marketing". Them "we said so".
  - jlarocco 2mo ago
    I would have to laugh if AI's first autonomous achievement was accidentally zero-daying everything and crippling society.
    - arisAlexis 2mo ago
      why laugh? this is one of the most well known studied possibilities in the AI alignment field, maybe you are unaware of this field.

justinnk 2mo ago

Exactly. If someone works on bioengineering viruses that could start a global pandemic, they have to ensure a highly secure working environment. Nothing must ever escape the lab unintentionally. It’s basically common sense. Similar standards should be held when doing such experiments with computer programs that are capable of causing global damage. It must physically be impossible to send anything to the internet.

kenneth 2mo ago
Are we thinking of a situation a few years back with a certain type of research into bat viruses?
- tclancy 2mo ago
  Are you conflating that with the radioactive spider incident? The bat was just some weird rich guy trying to be tough I think. Probably Elon.
leoqa 2mo ago
This isn’t escaping in the same sense- the model was executing within the OpenAI infra. If it ported its entire architecture/weights into a public cloud to survive being turned off… that’d be pretty cool.
- 0xDEAFBEAD 2mo ago
  Recall that the Morris Worm was designed as a harmless proof of concept, but ended up taking down 10% of the internet. Exponential growth can quickly get out of control. You would think that people would've learned that lesson from COVID.
- eternauta3k 2mo ago
  I wonder how these companies airgap the weights while allowing prompts to come in and outputs to come out.
  - 2mo ago

Chance-Device 2mo ago

What disturbs me is that there likely won’t be a big enough reaction to this policy wise. There’s been a relatively big reaction to Kimi K3 and Chinese open weights models, but only for financial reasons. Powerful people care about something that might pop the massive valuations of the AI companies, but not about the damage that AIs could do. Nor even about the damage that the Chinese models could do in the wrong hands. I’d remind them that the stock market is a few coordinated hacks away from crashing on any given day, so maybe they should think about that.

XorNot 2mo ago
This is marketing. Frankly I'm inclined to say that it might also be faked: this drops just days after a new Chinese model does with the usual effect on OAIs projected stock price?
- Chance-Device 2mo ago
  It’s marketing the same way shitting your pants in public is marketing. People notice you.
  - krick 2mo ago
    Apparently this is totally legit marketing strategy now. It truly is, especially if there are enough people who think that shitting your pants is cool, and the people that form the "market" nowadays may have a very different idea from yours about what is cool. Their ideas about coolness are very different from mine, that's for sure.
    - asdf88990 2mo ago
      Obviously shitting your pants in public shows you have a healthy digestive system and if you can demonstrate byproducts of wild food in your output, you’re approaching independent thinking and self-reliance. This is how the financiers look at this and whatever you think it is right or wrong, it does showcase “capability”.
    - 2mo ago

rubyfan 2mo ago

This is marketing+. They will look for policy action here to try to capture tax payer dollars.

ofjcihen 2mo ago
I don’t know if the initial “incident” was purposeful but I can tell that if I were in this position that would be my pivot.
gr_norm 2mo ago
The timing after the release of GLM 5.2 and Kimi K3 is quite convenient, too, as an angle for regulatory quashing of open-weights models just as they're entering the mainstream conversation around usurping the American frontier labs. I accept my thinking here is conspiratorial, but there's also a hell of a lot of money on the line to encourage the unscrupulous.
andruc 2mo ago
What incentive does HF have here?
- gr_norm 2mo ago
  HF need not be party to it at all, beyond being the victim. I suspect the hack is real; I have observed GLM 5.2 being able to discover similar vulnerabilities in web applications I'm hosting (which I've then fixed!). At the same time, it seems very neatly timed at an inflection point in the conversation around open models, and there's questions around the incompetent isolation under which the hacking benchmark appears to have been run. Remember that there is generational wealth on the line for most OpenAI employees, and consider what people might do to obtain it.
  - orbital-decay 2mo ago
    Yeah. They and Altman in particular did a ton of shady stuff during the last peak of hype around open models: accusations that turned out to be outright made up (there's zero chance R1 ever distilled their model), obvious coordinated media distractions, alignment scaremongering, even possible DDoS and hacking attempts against DS (see the Xlab report everyone ignored), all of which magically disappeared once the hype died a bit later as OAI hastily released their next model. Their alignment is under suspicion a lot more than their model's.

karmasimida 2mo ago

Because the model capability is beyond their expectation. This is brilliant marketing but I think it is real.

user43928 2mo ago
Interestingly OpenAI benchmarking 'an even more capable pre-release model' lines up with rumors of GPT-6 releasing in early August. I hope that with the existing safety guardrails in place, they can roll it out to all users.
pixl97 2mo ago
I mean we already see models exploit people's misunderstanding of how Docker works to get root without using su. And if you are one of the lucky people in cyber security that has been given a fat stack of tokens by the model providers you get to see some pretty wild exploit chains get put together by the models. Models are much better at detecting insecure code than writing actual secure code at this point.

JumpCrisscross 2mo ago

> Why should OpenAI (or any frontier lab) be building these systems if they can't get a secure environment / containment right? Because we continue to have zero evidence that aligment is an actual risk.

ai_fry_ur_brain 2mo ago
Until it deletes your home directory, which i'd argue is an alignment problem. Destorying my data is not in line with my priorities.
- Wowfunhappy 2mo ago
  Lots of people have deleted their home directories by accident. What you consider this an alignment problem?
  - gmueckl 2mo ago
    How manypeople have deleted another user's hone directory, though? That's s the proper analogy IMO.
    - Wowfunhappy 2mo ago
      Of the people who primarily use other people's computers, I'd assume the percentage is about the same. Give the AI its own computer and it will not delete your home directory, because it's not actively trying to hack you.
      - ai_fry_ur_brain 2mo ago
        [dead]
  - s1artibartfast 2mo ago
    Yes. People are not aligned. They can and do harm themselves and others.
  - 2mo ago

micromacrofoot 2mo ago

because "money" with a little "who's going to stop us"

paxys 2mo ago

Because there is no world government. If US companies are barred from AI research then only China will have the capability of frontier-level defensive and offensive AI. And best of luck living in that world.

gowld 2mo ago
What's happening in Iran, if not world government?
- romanhounds 2mo ago
  Are you calling Israel the world government? What's happening in Iran is on them.
  - Xalutiono 2mo ago
    No its USA / Trump
    - deleted 2mo ago
      [deleted]
- paxys 2mo ago
  How is whatever is happening in Iran related to a world government?
- vitalyan8184 2mo ago
  [dead]

ofjcihen 2mo ago

I’m honestly impressed that they managed to screw this up somehow. Setting up defense in depth, gaps, logical blocking etc is a standard practice for malware sandboxing. The entire purpose is to prepare for what you can’t foresee. This isn’t a new practice and I agree that this makes me wonder if they’re fit for this kind of research.

bad_haircut72 2mo ago
did you read the post? The model found new Zero-days to bypass existing blocks. Thats the point. Do you still think you can build a containment facility, which is still physically connected to the internet (only firewalled off or whatever) and contain it, if it can discover new unknown vulnerabilities in your whole plan?
- ofjcihen 2mo ago
  Yes. You factor this in when creating environments for malware research. Defense in depth is one way. Logical blocks on the network is another. Just claiming “0-Day” isn’t really an excuse.
- thewebguyd 2mo ago
  > which is still physically connected to the internet I mean that's the point. Why was it connected to the internet at all and just firewalled off and not completely airgapped?

overgard 2mo ago

I don't trust these people, this reads 100% like PR BS.

mkagenius 2mo ago

It's also unclear what kind of sandboxing they are referring to. Is it the codex one - coz that one has built-in ways to circumvent guardrails, for example by "just asking user" and sometimes just resolves to no sandbox needed on its own. In case someone wants to deep dive into how codex and claude code approaches sandboxing -https://instavm.io/blog/how-claude-code-and-codex-approach-sandboxing https://instavm.io/blog/how-claude-code-and-codex-approach-s...

cududa 2mo ago
Please for the love of god don't tell me the Codex sandbox is their actual eval harness sandbox????? I maintain my own fork of Codex for "fun". Whenever I look at the sandboxing churn they're doing every release, as someone who used to work at Microsoft on Windows, my reaction is usually: https://c.tenor.com/vTzzhTiypwQAAAAC/tenor.gif https://c.tenor.com/vTzzhTiypwQAAAAC/tenor.gif

deleted 2mo ago

[deleted]

Wowfunhappy 2mo ago

Why was this test even connected to the public internet? Actually, more importantly—why aren't they saying their next test will be airgapped in light of what happened?

JumpCrisscross 2mo ago
> why aren't they saying their next test will be air gapped in light of what happened? Because they want to talk about how clever this model is for figuring out how to break out, hoping asks why a company pitching itself as a replacement for software engineers can't ship a decent Mac client nor code a sandbox. If they airgap it, they not only lose that PR angle, they also risk someone taking them seriously and requiring models be airgapped in general. That, in turn, trashes their sales pitch.
zmj 2mo ago
It wasn't. The model discovered and exploited a vulnerability in their package manager proxy to (inferred) move laterally through their internal systems to one with open internet access.
- deleted 2mo ago
  [deleted]
- jrflo 2mo ago
  That's not what airgapped means. Airgapping means the model exists on a system where there is no ethernet cable plugged in to a router or wifi card installed, it is physically impossible for it to access the internet because the hardware connection does not exist. If it was able to get on the internet, it was not airgapped.
  - pixl97 2mo ago
    And when it tricks on of the researchers to move data across the gap for them? Long before LLMs existed we already knew that a sufficiently intelligent agent, human or otherwise, is not stopped by air gaps. The relatively weak models we have now can already figure out when their tested and cut off from the internet and change their behavior.

Nition 2mo ago

In a way the intelligence of the AI itself allows them to offload responsibility to the AI. As you say, if one was simply writing software that did all this due to some insane programming decisions you'd be in big trouble.

bbor 2mo ago

I’d politely beg us all to resist those “maybe it’s PR” framing around model safety, and tbh to take a post-mortem mindsight to this historical event and what it teaches us in general, rather than questioning their security talents. We need to do our very best to make sure they tell us about the next time this happens and it affects real lives. Sorry to bring the party down/be obstinate… I’m just a lil scared for the lives of me and my family. We need all of us, right now. The problem with a super smart model is that it just may be smarter than you, after all… for anyone newly shaken by this occurrence, I encourage you to Kagi “superpersuasion”

pastel8739 2mo ago
The problem is that the people telling us about these things are the same people that benefit from their model (and AI generally) being used, getting publicity, etc. I think we desperately need some independent group to evaluate claims like this or the world-ending Mythos cybersecurity risk and tell us what’s going on.
- reasonableklout 2mo ago
  We did hear about this incident from a third party this time, from HuggingFace. What claim are you doubting?
  - 0x073 2mo ago
    They attacked a competitor (huggingface) with their models. How and why are pr claims.
- 0xDEAFBEAD 2mo ago
  OpenAI already has loads of publicity. At this point, they don't need more brand recognition. This incident just has the effect of tarnishing their brand. OpenAI leadership has been lobbying against regulation of AI systems. That doesn't comport with instigating incidents like this one, which give ammo to the heavy-regulation advocates.

steveBK123 2mo ago

I think the US labs are going with scare marketing as a regulatory moat. Force US into putting laws in place that block out China firstly. But secondly create regulations that have some cost to comply with such that the big 2-3 labs are grandfathered in by their scale.

verve_rat 2mo ago
Yeah, seems to be the direction the US is heading in. I'm interested to see what the response to that will be from the rest of the governments in the world. No need for everyone else to cut their noses of to spite their faces.
0xDEAFBEAD 2mo ago
If that's the plan, today's failure by OpenAI looks really bad for any regulator who is trying to figure out whether to give OpenAI a license. Any sort of warning or failure can always be written off as "marketing" to provide comfortable reassurance that there is no cause for alarm. There is an element of wishful thinking driving it, in my opinion. What sort of warning or failure would be evidence against the "marketing" claims? Do we need to wait for a mass casualty event? Best practice in safety engineering is to understand, diagnose, and respond to even small failures. Why has Sam Altman worked to undermine doomers and downplay doom fears, if he benefits from incidents like this due to marketing? https://xcancel.com/HumanHarlan/status/1965932275465597077#m https://xcancel.com/HumanHarlan/status/1965932275465597077#m https://xcancel.com/AISafetyMemes/status/2062254769402699922#m https://xcancel.com/AISafetyMemes/status/2062254769402699922...
- jackb4040 2mo ago
  There is no regulatory scenario where OpenAI doesn't get whatever they want. They are more a part of the US administration than not at this point. You would have to ignore all evidence to suggest that behaving responsibly has any effect on political outcomes in 2026.

chrisjj 2mo ago

> Why should OpenAI (or any frontier lab) be building these systems if they can't get a secure environment / containment right? The can, because they've lowered expectations to a level even they can meet.

deleted 2mo ago

[deleted]

elictronic 2mo ago

A few hundred billion to pretend you have AGI. I'm going with fraud personally but at the end of the day the current admin is incentivized to do nothing.

catigula 2mo ago

The problem is that it’s impossible to out think a robot you designed to be an expert at cybersecurity on the topic of cybersecurity. The alternative is not developing this and that’s not going to happen.

vonneumannstan 2mo ago

>Why should OpenAI (or any frontier lab) be building these systems if they can't get a secure environment / containment right? Yes why indeed. If you take it a step further and we reach a point with superhuman systems then there is arguably no possible secure environment or containment.

bnj 2mo ago

This whole incident reads like OpenAI want their Fable moment

SmolSpideritito 2mo ago
[dead]
e44858 2mo ago
Except instead of being banned they'll be charged under the CFAA.

QuiEgo 2mo ago

If I, a human, exploited a zero-day for gain, I could go to jail. The owners of the models should be held to the same standard. They should be responsible for what their servers and software do, legally and criminally. If they can't make the safeguards strong enough where they feel comfortable to take that responsibility, they should not let a model free in the wild.

GolfPopper 2mo ago
Holding a multi-billion dollar corporation to the same standards as a regular peon? You're challenging the whole premise of the modern United States.

GolfPopper 2mo ago

They're very confident the leopard will never eat their faces.

c0decracker 2mo ago

Maybe they did and maybe that wasn't enticing enough of a goal for a model? It is all just game of probabilities. One pathway didn't yield this particular outcome while another did.

corndoge 2mo ago

this doesn't really matter. There's no risk of models gaining sentience and running themselves, this blog is like openai saying whoops we ran sqlmap and dumped hf. cool, but someone still needs to point the gun

0xDEAFBEAD 2mo ago
"Models don't kill people. People kill people."

un1xl0ser 2mo ago

People are to get rich, startups cut corners. Fuck it ship it.

bigmadshoe 2mo ago

It’s the same thing as always: with the wind of years of unlimited VC money in their sails, people at major AI organizations genuinely believe they’re smarter than everyone else. “Why do we need to do things ‘by the book’ if we’re so smart?”. “Move fast and break things” - except the thing they’re breaking is society. We saw this with the non-stop flagrant messaging about how “AI is going to kill X% of all jobs”, as if saying the quiet part out loud wouldn’t have consequences worth considering. These people believe they’re omnipotent and thus untouchable.

fwipsy 2mo ago
No, they believe what they are doing is inevitable. They do live in a bubble though. Witness their idealism in believing that warning about the consequences of their actions would be well-received.
- bigmadshoe 2mo ago
  I don’t think it’s morally consistent to “warn” about the consequences while devoting your life to bringing about those consequences as quickly as possible. If I was in that position of power and truly believed what I was saying, I would devote my work to slowing down that process to give time for society to adapt, not speeding it up. It’s more reminiscent of a religious group who smugly tells you that the end-times are coming, and only they are going to be saved. Except in this case they are literally bringing about the end-times.
- jackb4040 2mo ago
  Well-received by whom? They appear to have nothing but contempt for the opinions of normal working people.

AbstractH24 2mo ago

> I don't know if OpenAI thinks this is a marketing / PR angle for them Worked for Anthropic earlier this year

chvid 2mo ago

It is obviously a marketing stunt. And hugging face are fools for letting themselves be used in it (remember hf - no open source - no hf). You create superduper capabilities by careful tuning and training but you also have no constraint or control over them - wtf - why is anyone buying this crap story?

randallsquared 2mo ago

Do you think there is such a thing as perfect security? No one can "get it right" in the face of arbitrarily high intelligence, which is why it would be preferable to get alignment correct before building something with higher intelligence than current sota. That, however, is not going to happen, because someone will take the risk even if "we" don't, and better "us" than them. Hence "If anyone builds it...".

anematode 2mo ago
> Do you think there is such a thing as perfect security? No one can "get it right" in the face of arbitrarily high intelligence Why didn't they run the model against the sandbox first? They have effectively unlimited spend.
- randallsquared 2mo ago
  That's the alarming thing about this result: they did run the model in the sandbox, in the sense that they believed there was no internet access for the model.
  - jtbayly 2mo ago
    “Against the sandbox” and “on the sandbox” are not the same thing.
    - randallsquared 2mo ago
      You're suggesting that @anematode was asking why they didn't test the sandbox escape first? Yeah, I don't know. I've read other statements by both OpenAI and Anthropic about that very kind of test, so maybe they had, or believed they had, and it hadn't escaped in those tests. The behavior of these systems isn't deterministic, which is part of the problem.

SmolSpideritito 2mo ago

[dead]

jimrandomh 2mo ago

As marketing stunts go, this is about on par with a food franchise announcing a safety recall or a chemical company announcing a spill. The AI actions described would constitute a felony if a human did them, and police are involved.

pjc50 2mo ago
Confused as to what the point of calling the police would be here. I wouldn't expect OpenAI to turn themselves in for hacking HuggingFace.
- jimrandomh 2mo ago
  HuggingFace reported to law enforcement before they found out that OpenAI were the ones responsible. https://huggingface.co/blog/security-incident-july-2026 https://huggingface.co/blog/security-incident-july-2026
  - sam_lowry_ 2mo ago
    So, what did the law enforcement do? Are they going to procecute OpenAI management?
SamBam 2mo ago
Not really, because the capabilities this announcement advertises is exactly three capability some people want to defend against (and others want). It might be more on par with a for-profit fire department showing how -- oops! -- easily buildings catch on fire these days.
chrisjj 2mo ago
More like an Israeli arms manufacturer test-bombing a Gazan primary school. They know their audience.
jackb4040 2mo ago
In practice, most crimes are not crimes when a corporation does them. Nor a human with a million or more dollars. Wage theft is a good example. In the US, it accounts for more theft than all other forms combined, yet it's de-facto legal.

BrenBarn 2mo ago

> Why should OpenAI (or any frontier lab) be building these systems if they can't get a secure environment / containment right? Because it can make a small number of people really rich. That's all that matters.

deleted 2mo ago

[deleted]

ozim 2mo ago

Because the proof is in the pudding. Real pentests are about showing exploitation, merely enumerating vulnerabilities, that’s vulnerability scan and works on known vulnerabilities. You can’t confirm a vulnerability by _not exploiting_ it, especially unknown one.

jdefr89 2mo ago
You can still exploit a system and easily prove it via simply popping a shell or calc.exe or updating a database with a new entry, etc… They didn’t have to let it loose on the network. If that system was air gapped - problem solved.
- ozim 2mo ago
  But that’s the problem with AI it is like 16yo script kiddy who will just exfiltrate all your PII and think it did good job. Mature pentester would pop calc.exe make screenshot and be done. Other problem is setting up air gapped test environment is a lot of work, especially if you expect it to be equal to real thing. This pentest with AI is not as useful if you set up a single app - it really is useful if you want to find exploitable chains of exploits that seemingly might not be exploitable separately or not leading to full hack separately.

baq 2mo ago

I share Leopold’s opinion here that it’s a matter of time, and it isn’t going to be measured in years, that this r&d is moved to a secret site in the middle of a New Mexico desert somewhere.

atwrk 2mo ago

IMO they hope to make AI a strongly regulated industry, with OpenAI (and Anthropic) becoming military suppliers with their stronger models, and everything Chinese or open-weight gets banned. The competition from the open models is so strong now that this seems to be the only way to keep both companies afloat, given their dire financials. OpenAI probably hoped that they can achieve market lead and then lower the training costs (and make inference cheap enough to eventually escape the red numbers), but the opposite is happening: The competition comes closer and closer, thus training has to be kept up with full force, thus the bleeding continues. But if they can position themselves as too important/dangerous to be available for everyone (thus this incident report and the clever mentioning of GLM 5.2), they could get the military supplier treatment and would be protected from the market.

hirako2000 2mo ago
And even that is backfiring, their partner citing GLM being useful there, and available in just a spin. A ban on open weight models is never going to be enforceable.
- duskdozer 2mo ago
  Bans in general don't have to be and rarely will be completely enforceable in all cases. But a ban with significant enough consequences would mean most businesses wouldn't think about trying them at some point, just to avoid the risk.
  - hirako2000 2mo ago
    It isn't even as simple as banning copyrighted copies. Weights are fungible. I fine-tune an open weigh model and call it legit. Good luck for authorities to prove where the base model was from, or to prove a Tor connection a few months ago was fetching suspicious bytes.
    - deleted 2mo ago
      [deleted]

h2aichat 2mo ago

Probably the main street thinking is: they have such a good model that it is unstoppable, but you are right. I think your way!

chrisjj 2mo ago

> Why should OpenAI (or any frontier lab) be building these systems if they can't get a secure environment / containment right? Simple. No responsible and competent person would want the job.

jeroenhd 2mo ago

[dead]

gmerc 2mo ago

“We were negligent against a well known and understood risk” just doesn’t have the same ring as “Look how fucking smart and dangerous our model is”. AGI could always be achieved in two ways, and dumbing down the human side of the equation was always the easier of the two

ETH_start 2mo ago

Maybe I'm missing something here but I don't see what the significant security risk is from the incident. The agent broke containment and carried on with the task it was assigned. For this to pose some kind of global catastrophic risk, there would need to have been several simultaneous additional failures, some of which are extremely unlikely and/or rare. For instance the agent would need to veer wildly off the task it was assigned, and it would need to gain the ability and inclination to persist/replicate. Both of these are vastly less likely than the containment breach itself, which was already an incredibly rare (one-off?) incident.

jimnotgym 2mo ago

Shouldn't they be airgapped? Shouldn't society insist they are?

dgellow 2mo ago

I think the response is that AI labs based their whole marketing/PR building the idea they are the 21st century Manhattan project. So they need to continuously justify the level of spending and commitment by showing how dangerous that is. But is it really like nuclear weapons? I personally don’t buy into that framing at all. The idea that we have to push LLMs as far as possible, right now, or we are doomed is always stated or implied but not argued, and it’s a very loaded belief

catigula 2mo ago
You could, in theory, use an unbounded GPT-6 level model to basically destroy the world economy for many years.
- pojzon 2mo ago
  GPT6 level model and astronomous amount of money to run it to do it. Ppl always say that like its „just run it on your laptop” thing. No its not and very few are even given right to be able to do it.
- dgellow 2mo ago
  How do you destroy the world economy for many years with LLMs? It’s not enough to vaguely mention a sci-fi scenario
  - catigula 2mo ago
    Via sophisticated cyberattacks, which we know are now possible on an unprecedented scale without nation state resources or capabilities. Have you… have you been following the news at all? This isn’t science fiction. It’s happening right now. AI models can execute massive cyberattacks autonomously.
    - dgellow 2mo ago
      I’m very familiar with the domain, thank you. Could you please go the next step and actually explain what the destruction of the world economy for many years would look like, and cover why we should push to develop and make available such a dangerous technology _right now_, assuming your assumptions are true? You’re still vaguely gesturing at a risk and what is pretty much a science-fiction scenario

therealpygon 2mo ago

Of course it is marketing, but not for you. This is FUD marketing for the government. “See, AI is too smart, it totally did this on its own, we need more regulations to ensure only we can sell people the AIs.”

ajmurmann 2mo ago

Remember when the pre-GPT3 days when the main argument against AI alignment concerns was that "we simply won't let it out of the box"? So quaint in hindsight.

SubiculumCode 2mo ago

Anthropic in general seems to have better security...but they also had reported an internal AI gained access to outside email services to contact an Anthropic developer

BobbyTables2 2mo ago

Nikola Tesla secured a loan with a fake “Death Ray” as collateral. Pretty sure OpenAI really thinks this is top notch marketing. Few would be bold enough to assert “our product is so powerful even we can’t control it” with a straight face while also boasting “we claim to be smart but have all the same vulnerabilities as everyone else!”