4 ms·
Ironically Hugging Face had to use a Chinese model to stop a Rogue US AI, since the Guard Rails prevented them from using Sol or Fable to remediate this attack.
by NyxWulf 2mo ago
Ironically Hugging Face had to use a Chinese model to stop a Rogue US AI, since the Guard Rails prevented them from using Sol or Fable to remediate this attack. LOL
- pizlonator 2mo agoIncredible. I had to dig for the source: https://huggingface.co/blog/security-incident-july-2026 https://huggingface.co/blog/security-incident-july-2026 section “the asymmetry problem” Quote: “When we started the log analysis, we first used frontier models behind commercial APIs. This did not work: the analysis requires submitting large volumes of real attack commands, exploit payloads, and C2 artifacts, and these requests were blocked by the providers' safety guardrails, which cannot distinguish an incident responder from an attacker. We ran the forensic analysis instead on GLM 5.2, an open-weight model, on our own infrastructure. This had a second benefit: no attacker data, and none of the credentials it referenced, left our environment.”
- embedding-shape 2mo ago> This had a second benefit: no attacker data, and none of the credentials it referenced, left our environment.” Well, not none of it, to be entirely nitpicky, as they've already must have sent data at first to have received the rejections :) In the end, it ended up being OpenAI's agent actions anyways so doesn't really matter, and the credentials it seems like the agent also had gotten to those too already. Still, I'm sure they'll look differently at hosted/restricted models after this event, as will many others.
- reasonableklout 2mo agoInteresting that HuggingFace's disclosure was 5 days ago, it seems neither they nor OpenAI figured out it was an OpenAI model in evals until now
- deleted 2mo ago[deleted]
- tdiff 2mo agoWould be funny if the defending side sent all the info they have to openai, tipping off to attacking models that they were noticed.
- throwfaraway4 2mo agoIts almost too good
- neuroelectron 2mo agoOK, that's some interesting information but they used OpenAI without guard rails to pull off the attack so how did they do that? That's according to the article, so it kind of invalidates the point you're making.
- embedding-shape 2mo agoThe "malicious" agent was run by OpenAI and had access to models the public (or others outside of OpenAI as I understand it) doesn't have access to.
- paxys 2mo agoThe attacker (OpenAI) was using the model without guardrails. The defender (huggingface) did not have access to the top models so had to use weaker ones to detect the threat.
- neuroelectron 2mo agoRight, so they are using the full model that they rent out to intelligence agencies in the government, and presumably Israel
- segmondy 2mo agoJailbroken, all LLM models can be broken. ALL.
- Sol- 2mo agoPerhaps fortuitous timing for OpenAI that they can spin the fact that defenders have to resort to open Chinese models because OpenAI and Anthropic actively sabotage them with nerfed models into a nice message of making Huggingface part of the privileged group entitled to secure systems.
- vsgherzi 2mo agoAnother important part here. It's not as if they prompted the open source AI to stop the rogue AI but rather just used it as a tool to crawl logs and determine what happened.