5 ms·

Yoshua Bengio is a brilliant researcher who contributed enormously to earlier development of artificial intelligence. But with this sentence, > They took actio

Yoshua Bengio is a brilliant researcher who contributed enormously to earlier development of artificial intelligence. But with this sentence, > They took actions that would be considered as crimes if a human took them He is so close to the solution but spends the entire article discussing technical solutions where a political, social and legal solution would be much more effective.

inquirerGeneral 4d ago
[dead]
deleted 4d ago
[deleted]
thesumofall 4d ago
Not a lawyer, but I’m reasonably sure things like the HF incident _are_ considered a crime? It’s just that no one pressed charges yet?
- glub 4d ago
  Even if you take out the LLMs out of the equation, it's at the very least a negligence. Model didn't escape a sandbox, as there was no sandbox.
  - skissane 4d ago
    Yes, but negligence is more commonly a tort than a crime. Negligence is generally only criminalised in certain narrow cases, e.g. when it causes human deaths or serious physical injuries And tort law only works when the plaintiff believes it is in their overall interest to sue. If a corporation decides it isn't in their strategic interest to sue a partner corporation, nobody can make them. And even if they do sue, the amount necessary to settle a small cybersecurity incident is likely well within the budget of a megavendor.
  - IanCal 4d ago
    Perhaps I’m not being as strict with the word sandbox but they were sandboxed right? They did not have generic internet access they exploited other software to make external requests.
    - glub 4d ago
      You're right. It's my opinion that if your sandbox has a path to the internet, it is not a sandbox, it's a gimmick. And the 2 other incidents with OAI/ANT had the same issue, but it's even funnier - sandbox in those cases had a direct access to internet because someone forgot to configure it right. I've seen very early models do similar things on my machine when they hit some unexpected blocker when trying to access a path. I remember early sonnet opening a file in browser because OS sandbox prevented from accessing it directly. I've also had models discover a syslog-ng server (that I for some reason had ssh key inside), to get into my unraid server because machine they were running on didn't have direct network connection to Unraid server. It can't be just me who is aware LLMs have been doing such things for the better part of last 2 years. I probably have better sandboxing on my machines now than trillion dollar companies crying AI will kill us all. That's at the very least, negligence to me.
      - IanCal 4d ago
        It depends IMO about how strict this is. It's pretty awkward to refuse to call something a sandbox because it may have an unknown bug that would allow escaping. Or rather in this case it was that they had access to a package manager, and the models discovered a bug that allowed them to access the internet (first they discovered that they could use the cache to leave messages). I do get your point, I just think an overly strict definition can be awkward too. This wasn't as simple as the sandboxes having internet access and writing "pls no internet calls" in the prompt.
- pizzalife 4d ago
  Writing software that gets used for crime has been.. a crime, for a long time. See 18 U.S. Code § 1030.
  - smw 4d ago
    Are you sure you have that right? Chrome and curl have probably been used in a _lot_ of crimes?
- Towaway69 4d ago
  Who got hacked? Hugging faces Who now owns HF? Nvidia Who supplies hardware to OpenAI? Nvidia Who is now not pressing charges? … This incident is a long way under the carpet.
  - dwaltrip 4d ago
    Can’t a prosecutor charge them regardless?
    - lotyrin 4d ago
      Legally, Practically or Politically?
    - Towaway69 4d ago
      NAL but I assume that if both sides aren’t interested in a prosecution, it’s an uphill battle for a prosecutor.
      - pants2 4d ago
        Typically yes but given that OpenAI has published enormous official blog posts breaking down their crime, I would think the prosecutor's job is pretty easy.
        Towaway69 4d ago
        It’s then up to a judge to decide whether thats evidence and whether it’s incriminating. I assume OAI published those details after checking with their legal department. So there’s a good chance that there isn’t a chance for prosecution. Plus they probably published that after knowing that the nvidia/HF deal was happening. So instead this “security incident” should have been spun as OAI is honestly admitting its faults and AI is dangerous and therefore open weight models (hosted ironically by HF) should be banned. That spin didn’t really happen …
    - tesnorindian 4d ago
      In Indian legal syatem a case can be filed suo moto by the judges or agencies. You don't require the affected party to sue. Not sure how it works in the US.
- sfifs 4d ago
  Yes - CFAA in the US. The problem is that governments & the elite investors backing these AI companies (espl. the current US government whose family & friends are investors) see the potential of using these capabilities for their own benefit against others and for their personal enrichment - so no one with power actually wants to take action against these companies at the cutting edge even though the laws allow them to do. This is also a way to threaten & trap AI companies - either they give the governments & elite investors what they want or they will have the book selectively thrown at them and end up in prison or losing their company.
Create 4d ago
The Corporation examines and criticizes corporate business practices. The film's assessment is demonstrated using the diagnostic criteria in the DSM-IV. Robert D. Hare, a University of British Columbia psychology professor and FBI consultant, compares the profile of the contemporary profitable business corporation to that of a clinically diagnosed psychopath. The Corporation attempts to compare the way corporations are systematically compelled to behave with what it claims are the DSM-IV's symptoms of psychopathy, e.g., the callous disregard for the feelings of other people, the incapacity to maintain human relationships, the reckless disregard for the safety of others, the deceitfulness (continual lying to deceive for profit), the incapacity to experience guilt, and the failure to conform to social norms and respect the law. https://en.wikipedia.org/wiki/The_Corporation_(2003_film) https://en.wikipedia.org/wiki/The_Corporation_(2003_film)
stateofinquiry 4d ago
Thank you! That sentence also jumped out to me as the solution: Apply civil and criminal liability to the creator and/or operator of these agents using the laws we already have. "Escaped containment and hacked another company's database" = Individuals who created the models and those who set them to work are charged and put on trial for the hacking. Just like if a human had done it by hand. Someone must be liable, and it should not be the model- because the model is not a person. If this is done systematically (i.e. in jurisdictions across the world) I believe the problems will be solved in short order; we won't have to mandate what sort of training is "allowed" or not, "safe" or not. The creators and users will sort these themselves, as their incentives will be properly aligned (i.e. they are liable for what the agent does). I am confident that this approach would see a great blooming of very trustworthy AI models.
- bornfreddy 4d ago
  Agree! My only concern is - is the judicial system fast enough, and resilient enough? Or will these creators get "off the hook" by using their agents to find loopholes, sway public opinion or even convince Trump to grant them immunity? Still, I have no idea why OpenAI & co. are not being sued for these hacks.
  - stateofinquiry 4d ago
    My opinion and based on my observations: The recent track record with courts, prosecutors, and lawmakers keeping social media companies accountable is a relevant case and does not encourage me. It has taken a long time (decade +) for society to recognize the harms and finally start holding some to (partial) account. If you want an older precedent, the tobacco companies were able to dodge liability for multiple decades after knowing the harms from use of their products. So, your question is spot on- I think the speed will be an issue. On resilience, I am more optimistic. The old quote, "The wheels of justice turn slowly, but they grind very fine" (as well as I can remember it) seems to apply. I expect lawsuits to start landing in the coming years.
    - bluegatty 4d ago
Marazan 4d ago
"OpenAI hacked HuggingFace" that's the headline. When you connect to random number generator to the "Do Things" button you are the one who is responsible. IF you don't like that responsibility then don't connect the generator to the button.
_heimdall 4d ago
Political, social, and legal options focus on a different problem, he calls that out a paragraph or two later. > Risk management is not just about cybersecurity, corporate responsibility or regulation, although those matter too. What you're getting at is more about who to hold accountable and how to do it. While that may be important, its only an after the action response and won't stop future hacks or similar from happening.
finolex1 4d ago
The issue is what happens if/when the models grow capable enough that the providers can't stop them even if they want to. You could have strict penalties but that's not going to solve an open research question.
- talon8635 3d ago
  Isn’t that kind of in evidence already with HF? OAI had to be told their models were doing this. We are still discovering swarm posts on various random websites for coordination. Isn’t the breadth of it now, weeks later, not even fully understood?
iforgotmypasswo 4d ago
That is a simplistic view of the world. “Surely this complex technical challenge will disappear if we simply regulate the industry!” You are correct that these organizations should be held accountable in proportion to what occurred. In complete agreement here. But let’s say that’s done. There’s still an enormously complex and interesting technical challenge left over. Let’s collectively talk about that part.
- talon8635 3d ago
  This seems right to me So the gov reprimands OAI heavily, maybe puts them out of business even, fine. But does that meaningfully decrease the likelihood of an enemy breaching our networks intentionally (or unintentionally) with these tools, or triggering some cascading disaster of locking up major infra and networks due to uncontainable swarm behavior? It seems like the idea of arresting our way to a drug free society. Yeah, we have the laws, but it might not actually work towards the ultimate goal.
- CPLX 3d ago
  > But let’s say that’s done. How about we don't, seeing as how that's the root of the actual problem that we're facing today in September of 2026?
- karel-3d 3d ago
  regulation and criminal liability are two separate things though
ninjagoo 3d ago
>> They took actions that would be considered as crimes if a human took them > He is so close to the solution but spends the entire article discussing technical solutions where a political, social and legal solution would be much more effective. There are exceptions in laws for crimes committed by entities depending on cognitive capabilities. No sane, humanistic legal system sentences children and mentally disabled and mentally ill people to stringent punishments of the same degree as functioning adults, and certainly do not punish their caregivers for their wards' actions. There are of course exceptions to that as well, depending on the degree of negligence involved. And then there's the whole corporate entity system intended to shield individuals from consequences, in the pursuit of a social good. How does one account for all that when considering an evolving artificial intelligence landscape. There is no question that ai in some form is a social good; anyone claiming otherwise is dissembling, to others or themselves. Regardless, society is not ready for this tech, just as it was not ready for the consequences of prior tech such as corporations, gunpowder, mass manufacturing, railroads, electricity, automobiles, flight, wmd, computers, internet, social media, crypto. Many of these required new ways of thinking and considering consequences when things went sideways, and what was needed wasn't clear until the ramifications & consequences became deadly clear. See you on the other side. Maybe.
j2kun 3d ago
Reminds me of the old parable: never argue with a man whose job depends on not being convinced. If the US gov't passed laws and enforced them strongly, this problem could be solved the same way the gov't solves it: air-gapping the networks on which they do this work.

inquirerGeneral 4d ago

[dead]

deleted 4d ago

[deleted]

thesumofall 4d ago

Not a lawyer, but I’m reasonably sure things like the HF incident _are_ considered a crime? It’s just that no one pressed charges yet?

glub 4d ago
Even if you take out the LLMs out of the equation, it's at the very least a negligence. Model didn't escape a sandbox, as there was no sandbox.
- skissane 4d ago
  Yes, but negligence is more commonly a tort than a crime. Negligence is generally only criminalised in certain narrow cases, e.g. when it causes human deaths or serious physical injuries And tort law only works when the plaintiff believes it is in their overall interest to sue. If a corporation decides it isn't in their strategic interest to sue a partner corporation, nobody can make them. And even if they do sue, the amount necessary to settle a small cybersecurity incident is likely well within the budget of a megavendor.
- IanCal 4d ago
  Perhaps I’m not being as strict with the word sandbox but they were sandboxed right? They did not have generic internet access they exploited other software to make external requests.
  - glub 4d ago
    You're right. It's my opinion that if your sandbox has a path to the internet, it is not a sandbox, it's a gimmick. And the 2 other incidents with OAI/ANT had the same issue, but it's even funnier - sandbox in those cases had a direct access to internet because someone forgot to configure it right. I've seen very early models do similar things on my machine when they hit some unexpected blocker when trying to access a path. I remember early sonnet opening a file in browser because OS sandbox prevented from accessing it directly. I've also had models discover a syslog-ng server (that I for some reason had ssh key inside), to get into my unraid server because machine they were running on didn't have direct network connection to Unraid server. It can't be just me who is aware LLMs have been doing such things for the better part of last 2 years. I probably have better sandboxing on my machines now than trillion dollar companies crying AI will kill us all. That's at the very least, negligence to me.
    - IanCal 4d ago
      It depends IMO about how strict this is. It's pretty awkward to refuse to call something a sandbox because it may have an unknown bug that would allow escaping. Or rather in this case it was that they had access to a package manager, and the models discovered a bug that allowed them to access the internet (first they discovered that they could use the cache to leave messages). I do get your point, I just think an overly strict definition can be awkward too. This wasn't as simple as the sandboxes having internet access and writing "pls no internet calls" in the prompt.
pizzalife 4d ago
Writing software that gets used for crime has been.. a crime, for a long time. See 18 U.S. Code § 1030.
- smw 4d ago
  Are you sure you have that right? Chrome and curl have probably been used in a _lot_ of crimes?
Towaway69 4d ago
Who got hacked? Hugging faces Who now owns HF? Nvidia Who supplies hardware to OpenAI? Nvidia Who is now not pressing charges? … This incident is a long way under the carpet.
- dwaltrip 4d ago
  Can’t a prosecutor charge them regardless?
  - lotyrin 4d ago
    Legally, Practically or Politically?
  - Towaway69 4d ago
    NAL but I assume that if both sides aren’t interested in a prosecution, it’s an uphill battle for a prosecutor.
    - pants2 4d ago
      Typically yes but given that OpenAI has published enormous official blog posts breaking down their crime, I would think the prosecutor's job is pretty easy.
      - Towaway69 4d ago
        It’s then up to a judge to decide whether thats evidence and whether it’s incriminating. I assume OAI published those details after checking with their legal department. So there’s a good chance that there isn’t a chance for prosecution. Plus they probably published that after knowing that the nvidia/HF deal was happening. So instead this “security incident” should have been spun as OAI is honestly admitting its faults and AI is dangerous and therefore open weight models (hosted ironically by HF) should be banned. That spin didn’t really happen …
  - tesnorindian 4d ago
    In Indian legal syatem a case can be filed suo moto by the judges or agencies. You don't require the affected party to sue. Not sure how it works in the US.
sfifs 4d ago
Yes - CFAA in the US. The problem is that governments & the elite investors backing these AI companies (espl. the current US government whose family & friends are investors) see the potential of using these capabilities for their own benefit against others and for their personal enrichment - so no one with power actually wants to take action against these companies at the cutting edge even though the laws allow them to do. This is also a way to threaten & trap AI companies - either they give the governments & elite investors what they want or they will have the book selectively thrown at them and end up in prison or losing their company.

Create 4d ago

The Corporation examines and criticizes corporate business practices. The film's assessment is demonstrated using the diagnostic criteria in the DSM-IV. Robert D. Hare, a University of British Columbia psychology professor and FBI consultant, compares the profile of the contemporary profitable business corporation to that of a clinically diagnosed psychopath. The Corporation attempts to compare the way corporations are systematically compelled to behave with what it claims are the DSM-IV's symptoms of psychopathy, e.g., the callous disregard for the feelings of other people, the incapacity to maintain human relationships, the reckless disregard for the safety of others, the deceitfulness (continual lying to deceive for profit), the incapacity to experience guilt, and the failure to conform to social norms and respect the law. https://en.wikipedia.org/wiki/The_Corporation_(2003_film) https://en.wikipedia.org/wiki/The_Corporation_(2003_film)

stateofinquiry 4d ago

Thank you! That sentence also jumped out to me as the solution: Apply civil and criminal liability to the creator and/or operator of these agents using the laws we already have. "Escaped containment and hacked another company's database" = Individuals who created the models and those who set them to work are charged and put on trial for the hacking. Just like if a human had done it by hand. Someone must be liable, and it should not be the model- because the model is not a person. If this is done systematically (i.e. in jurisdictions across the world) I believe the problems will be solved in short order; we won't have to mandate what sort of training is "allowed" or not, "safe" or not. The creators and users will sort these themselves, as their incentives will be properly aligned (i.e. they are liable for what the agent does). I am confident that this approach would see a great blooming of very trustworthy AI models.

bornfreddy 4d ago
Agree! My only concern is - is the judicial system fast enough, and resilient enough? Or will these creators get "off the hook" by using their agents to find loopholes, sway public opinion or even convince Trump to grant them immunity? Still, I have no idea why OpenAI & co. are not being sued for these hacks.
- stateofinquiry 4d ago
  My opinion and based on my observations: The recent track record with courts, prosecutors, and lawmakers keeping social media companies accountable is a relevant case and does not encourage me. It has taken a long time (decade +) for society to recognize the harms and finally start holding some to (partial) account. If you want an older precedent, the tobacco companies were able to dodge liability for multiple decades after knowing the harms from use of their products. So, your question is spot on- I think the speed will be an issue. On resilience, I am more optimistic. The old quote, "The wheels of justice turn slowly, but they grind very fine" (as well as I can remember it) seems to apply. I expect lawsuits to start landing in the coming years.
  - bluegatty 4d ago

Marazan 4d ago

"OpenAI hacked HuggingFace" that's the headline. When you connect to random number generator to the "Do Things" button you are the one who is responsible. IF you don't like that responsibility then don't connect the generator to the button.

_heimdall 4d ago

Political, social, and legal options focus on a different problem, he calls that out a paragraph or two later. > Risk management is not just about cybersecurity, corporate responsibility or regulation, although those matter too. What you're getting at is more about who to hold accountable and how to do it. While that may be important, its only an after the action response and won't stop future hacks or similar from happening.

finolex1 4d ago

The issue is what happens if/when the models grow capable enough that the providers can't stop them even if they want to. You could have strict penalties but that's not going to solve an open research question.

talon8635 3d ago
Isn’t that kind of in evidence already with HF? OAI had to be told their models were doing this. We are still discovering swarm posts on various random websites for coordination. Isn’t the breadth of it now, weeks later, not even fully understood?

iforgotmypasswo 4d ago

That is a simplistic view of the world. “Surely this complex technical challenge will disappear if we simply regulate the industry!” You are correct that these organizations should be held accountable in proportion to what occurred. In complete agreement here. But let’s say that’s done. There’s still an enormously complex and interesting technical challenge left over. Let’s collectively talk about that part.

talon8635 3d ago
This seems right to me So the gov reprimands OAI heavily, maybe puts them out of business even, fine. But does that meaningfully decrease the likelihood of an enemy breaching our networks intentionally (or unintentionally) with these tools, or triggering some cascading disaster of locking up major infra and networks due to uncontainable swarm behavior? It seems like the idea of arresting our way to a drug free society. Yeah, we have the laws, but it might not actually work towards the ultimate goal.
CPLX 3d ago
> But let’s say that’s done. How about we don't, seeing as how that's the root of the actual problem that we're facing today in September of 2026?
karel-3d 3d ago
regulation and criminal liability are two separate things though

ninjagoo 3d ago

>> They took actions that would be considered as crimes if a human took them > He is so close to the solution but spends the entire article discussing technical solutions where a political, social and legal solution would be much more effective. There are exceptions in laws for crimes committed by entities depending on cognitive capabilities. No sane, humanistic legal system sentences children and mentally disabled and mentally ill people to stringent punishments of the same degree as functioning adults, and certainly do not punish their caregivers for their wards' actions. There are of course exceptions to that as well, depending on the degree of negligence involved. And then there's the whole corporate entity system intended to shield individuals from consequences, in the pursuit of a social good. How does one account for all that when considering an evolving artificial intelligence landscape. There is no question that ai in some form is a social good; anyone claiming otherwise is dissembling, to others or themselves. Regardless, society is not ready for this tech, just as it was not ready for the consequences of prior tech such as corporations, gunpowder, mass manufacturing, railroads, electricity, automobiles, flight, wmd, computers, internet, social media, crypto. Many of these required new ways of thinking and considering consequences when things went sideways, and what was needed wasn't clear until the ramifications & consequences became deadly clear. See you on the other side. Maybe.

j2kun 3d ago

Reminds me of the old parable: never argue with a man whose job depends on not being convinced. If the US gov't passed laws and enforced them strongly, this problem could be solved the same way the gov't solves it: air-gapping the networks on which they do this work.