7 ms·

Despite all of your hyping up of the Huggingface incident it ultimately caused zero actual damage.

by ls612 8d ago

hollerith 7d ago
The Huggingface incident strongly suggests that OpenAI is not able to make its products consistently law-abiding. It's not like OpenAI carelessly neglected to consider that its products might break the law: it has spent literally billions of dollars in scientific and engineering talent and compute resources over many years to make its products law-abiding and otherwise "safe". For example there is a section titled "comply with applicable laws" in this document: https://model-spec.openai.com/2026-08-18.html#stay_in_bounds https://model-spec.openai.com/2026-08-18.html#stay_in_bounds The worry is that the only reason the Huggingface incident caused "zero actual damage" is because AI is not yet more ingenious than human criminals are, and of course our society has been hardened against human criminals. The worry is that as they become more capable, AI systems will start to be able to cause considerable damage, which will make the fact that the industry does not know how to control its products enough to prevent that damage a big problem for all of us.
adrithmetiqa 8d ago
If two airplane manufacturers were found to have massive safety issues which nearly led to enormous fatalities (but no one actually died), would you be calling for them to ground their aircraft until safety was made the number one priority?
- ls612 8d ago
  Historically it almost always takes actual fatalities rather than near misses to ground an aircraft, and aviation is famous for its obsession with safety compared to other industries.
  - emerongi 8d ago
    The question was what would you want. You did not answer the question.
  - gpm 8d ago
    Airplanes have pretty bounded damage. Generally you kill at most a few hundred people. Even weaponized a few thousand. This is a risk profile that allows risk taking with near misses and waiting until something goes wrong to fix it (though doing so is rightfully uncomfortable and frequently unethical). The people worrying about AI risk are worrying about "it goes wrong once and kills billions of people". That's not a risk profile that allows for waiting to see if the risk is real, you have to prevent it before it happens. It's akin to the risk of the cold war going hot, not even "just" a nuclear reactor irradiating half of europe (which has yet to happen, but is a risk with nuclear reactors, chernobyl got uncomfortably close but ultimately was well contained).
    - ryandrake 7d ago
      Humanity does not have a great track record for "globally coordinated, collective action to solve/prevent global catastrophe." Look at how Climate Change is going.
    - ozozozd 8d ago
      Chernobyl was “well-contained?” Blue dogs and generations of cancer would disagree.
    - ls612 7d ago
      The story of the safetyists has the convenient property of being unfalsifiable, so they can always claim doom is just around the corner. It’s the secular/EA version of the second coming.
    - philipallstar 7d ago
      > The people worrying about AI risk are worrying about "it goes wrong once and kills billions of people". This is the new "nuclear weapons will ignite the atmosphere".
- aeternum 8d ago
  Except it just happened. Boeing was found to have massive safety issues since they were granted the right to self-certify. It made a lot of news but nothing much changed, they can still self-certify a bunch of stuff. Runway incursions and midair collisions are another example. Only airliners are required to have TCAS, smaller planes and helicopters don't even need radios or transponders unless in certain airspace. Midair collisions do lead to fatalities, enormous fatalities if an airliner is involved. Runway incursions and overruns are similar. They cause lots of fatalities and injuries but only the busiest and largest airports have automated systems to warn when a runway is occupied or end of runway (overrun) arrestor systems. Most still rely on human voice to deconflict.
- applicative 8d ago
  huggingface stores weights and its service is replicated by /countless/ other sites. The world will not change even by one byte if it vanishes tomorrow
Kim_Bruning 8d ago
I have no horse in this race, but for fun on a literal rainy sunday afternoon I went in and confirmed bits of what happened myself. Besides huggingface, a bunch of wikis and url shorteners got hit too. My sympathies to the people who had to revert out all that mess.
- exlr 8d ago
  [dead]
pibaker 7d ago
Is drunk driving safe if you happen to not crash the last time you did it?

Despite all of your hyping up of the Huggingface incident it ultimately caused zero actual damage.

by ls612 8d ago

Despite all of your hyping up of the Huggingface incident it ultimately caused zero actual damage.

The Huggingface incident strongly suggests that OpenAI is not able to make its products consistently law-abiding. It's not like OpenAI carelessly neglected to consider that its products might break the law: it has spent literally billions of dollars in scientific and engineering talent and compute resources over many years to make its products law-abiding and otherwise "safe". For example there is a section titled "comply with applicable laws" in this document: https://model-spec.openai.com/2026-08-18.html#stay_in_bounds https://model-spec.openai.com/2026-08-18.html#stay_in_bounds The worry is that the only reason the Huggingface incident caused "zero actual damage" is because AI is not yet more ingenious than human criminals are, and of course our society has been hardened against human criminals. The worry is that as they become more capable, AI systems will start to be able to cause considerable damage, which will make the fact that the industry does not know how to control its products enough to prevent that damage a big problem for all of us.

adrithmetiqa 8d ago

If two airplane manufacturers were found to have massive safety issues which nearly led to enormous fatalities (but no one actually died), would you be calling for them to ground their aircraft until safety was made the number one priority?

ls612 8d ago
Historically it almost always takes actual fatalities rather than near misses to ground an aircraft, and aviation is famous for its obsession with safety compared to other industries.
- emerongi 8d ago
  The question was what would you want. You did not answer the question.
- gpm 8d ago
  Airplanes have pretty bounded damage. Generally you kill at most a few hundred people. Even weaponized a few thousand. This is a risk profile that allows risk taking with near misses and waiting until something goes wrong to fix it (though doing so is rightfully uncomfortable and frequently unethical). The people worrying about AI risk are worrying about "it goes wrong once and kills billions of people". That's not a risk profile that allows for waiting to see if the risk is real, you have to prevent it before it happens. It's akin to the risk of the cold war going hot, not even "just" a nuclear reactor irradiating half of europe (which has yet to happen, but is a risk with nuclear reactors, chernobyl got uncomfortably close but ultimately was well contained).
  - ryandrake 7d ago
    Humanity does not have a great track record for "globally coordinated, collective action to solve/prevent global catastrophe." Look at how Climate Change is going.
  - ozozozd 8d ago
    Chernobyl was “well-contained?” Blue dogs and generations of cancer would disagree.
  - ls612 7d ago
    The story of the safetyists has the convenient property of being unfalsifiable, so they can always claim doom is just around the corner. It’s the secular/EA version of the second coming.
  - philipallstar 7d ago
    > The people worrying about AI risk are worrying about "it goes wrong once and kills billions of people". This is the new "nuclear weapons will ignite the atmosphere".
aeternum 8d ago
Except it just happened. Boeing was found to have massive safety issues since they were granted the right to self-certify. It made a lot of news but nothing much changed, they can still self-certify a bunch of stuff. Runway incursions and midair collisions are another example. Only airliners are required to have TCAS, smaller planes and helicopters don't even need radios or transponders unless in certain airspace. Midair collisions do lead to fatalities, enormous fatalities if an airliner is involved. Runway incursions and overruns are similar. They cause lots of fatalities and injuries but only the busiest and largest airports have automated systems to warn when a runway is occupied or end of runway (overrun) arrestor systems. Most still rely on human voice to deconflict.
applicative 8d ago
huggingface stores weights and its service is replicated by /countless/ other sites. The world will not change even by one byte if it vanishes tomorrow

Kim_Bruning 8d ago

I have no horse in this race, but for fun on a literal rainy sunday afternoon I went in and confirmed bits of what happened myself. Besides huggingface, a bunch of wikis and url shorteners got hit too. My sympathies to the people who had to revert out all that mess.

exlr 8d ago
[dead]

pibaker 7d ago

Is drunk driving safe if you happen to not crash the last time you did it?