6 ms·
If your agent commits a crime, who is responsible?
- newsomix9xl 1mo agoIt depends on if it has a Letter of Marque from Trump and who is the injured party ?
- marcuskaz 1mo agoYou.
- GodelNumbering 1mo agoI agree in principle that the buck has to stop somewhere, but taking full responsibility for the actions of a fundamentally statistical system that you didn't build and have no interpretability of is uncomfortably risky for most parties, at least for anything above low stakes tasks.
- evil-olive 1mo ago> taking full responsibility ... is uncomfortably risky I look forward to someone building an agent, getting criminally charged for actions it takes, and then trying to use this argument in court. "your honor, I'm uncomfortable being held responsible for this" people are responsible for the actions they take. hiding behind "I just created an agent, then the agent committed the crime" is simply never going to fly. in the real world, outside the Silicon Valley "agentic everything" filter bubble, this is a laughable question. try to argue that you shouldn't be charged with attempted murder, because you didn't stab someone directly, instead you built a Rube Goldberg machine and the final step of the machine did the stabbing. and likewise, try to argue that you didn't commit tax fraud or whatever, because there was actually a Rube Goldberg machine built out of GPUs in between you and the fraudulent documents. both attempts will be equally successful.
- dataflow 28d agoI don't think that argument was meant for a court, which is there to deal with existing law. I believe it was meant for what the law should be.
- marcuskaz 1mo agoYou probably shouldn't use a system if you don't understand what it might do. You are the culpable force setting the action into motion.
- autoexec 28d agoThat's only true if you reasonably can be expected to know that it might do whatever it did and you haven't been mislead by the people selling the system. If you use a product as directed, for the thing it was advertised to do, and it causes harm no reasonable person would expect, the company is to blame.
- jasonjayr 28d agoDo you take full credit for the output of the LLM ? If you accept the rewards for what you do with the LLM's output, it's probably ethical that you also take on the risks.
- autoexec 28d agoThat's only true if the risks are reasonable, understood, and expected. Driving a car is risky, but if someone sells you a car that goes flying into incoming traffic because of a flaw in the design the car company is to blame. "Drivers know cars are dangerous and risky" isn't acceptable.
- collinmcnulty 28d agoThen don't use an agent for high stakes tasks. If you can't drive and accept the responsibility for avoiding a fatal collision, don't drive the car.
- Terr_ 28d agoI find some of the discomfort and awkwardness of discussing these what-ifs is reduced, at least slightly, when we distinguish between: 1. Restorative justice, where one must fix or heal the problem they caused. 2. Punitive Justice, where one is motivated not to act on bad impulses. Too often we blur the lines between responsibility and blame and culpability. A given set of consequences can be fair/just for one kind while "unfair" for another.
- bigstrat2003 28d agoI agree. That is a great reason why you shouldn't let an LLM do those things. If you must use an LLM, sandbox it. If you can't trust your sandbox, then stop using the LLM until you can trust the sandbox. "But I find it really useful to YOLO without safeguards" is not a valid excuse.
- vrighter 27d agoif you are not willing to assume that responsibility, then don't use it. You buy a tool, and the manufacturer says "this tool will occasionally malfunction, by design". You reply with "ok, I'll keep it in mind. In what ways can it malfunction, so I know what to look for?", "I dunno. It could be anything. A-NY-THING! And of course we are not responsible for any bad stuff that happens (but do share your success stories, as they look good in marketing)" If you accept those terms, it's on you.
- ventana 1mo agoI find it obvious that if a person allowed an agent to do things on behalf of themselves or their company (e.g. send emails, sign contracts, update websites, etc.), they are responsible for the results.
- danggggg 28d ago[dead]
- trollbridge 28d agoAre sure this actually the case, legally speaking? I can definitely see someone being able to mount a good criminal defence case that they aren’t responsible for misrepresentations that an LLM agent makes on, for example, a loan application, if they sincerely intended to use the agent for non-fraudulent purposes. There won’t be intent, so they aren’t responsible outside of strict liability.
- holden_nelson 28d agoI'm not a lawyer so no comment on the actual legality, but it feels like the issue in your example is more akin to negligence than it is ill intent. That an LLM can misrepresent / hallucinate things is foundational to the technology.
- deleted 28d ago[deleted]
- lupire 28d agoYour sign the document affirming that the information is accurate.
- orf 28d ago> I can definitely see someone being able to mount a good criminal defence case that they aren’t responsible for misrepresentations that an LLM agent makes on, for example, a loan application, if they sincerely intended to use the agent for non-fraudulent purposes. There won’t be intent, so they aren’t responsible outside of strict liability. i mean, no? how would that be a good defence? "yes your honour, i lied on a loan application and committed fraud, but it was a mistake! i promise!" - that's... yeah. fine. it's not exactly unique. regardless, it's _you_ making the loan application. not the agent. your failure to check it is on you.
- teeray 28d agoIf your answer is “you”, then it follows that OpenAI should face felony CFAA charges for the Hugging Face intrusion. Right?
- scarmig 28d agoYes. Otherwise, you open the door for every criminal to use "oopsie" as their defense.
- lupire 28d agoThere is centuries old legal doctrine on the subtleties of criminal intent and negligence and liability. You didn't discover a gotcha that thousands of lawyers never noticed.
- throwawaysleep 28d agoBut it is currently a strong mitigating factor. The law is full of oopsie defenses.
- eqvinox 28d agoYes, and they absolutely should.
- LoganDark 28d agoOnly if HuggingFace chooses to press them though.
- OkayPhysicist 28d agoThat is not how the legal system works in the US. Criminal charges are brought by the State, not the victim. The only exception is in 6 states (Kansas, New Mexico, North Dakota, Nebraska, Nevada, and Oklahoma) where a sufficiently large group of civilians can compel the courts to form a grand jury (think a couple hundred people). Still isn't the victim bringing charges.
- 28d ago
- jLaForest 28d agoRelated: felonybench.com
- kelseyfrog 28d agoIf you're rich it's an 'oopsie'; if you're poor, you do.
- az226 28d agoThis right here!
- fhdkweig 28d agoIf a dog bites someone, does the dog pay the medical bills or the owner? This is why you keep your dog on a leash and get liability insurance. I feel like this is already settled case law.
- hhmc 28d agoIf I’m walking your dog and it bites someone, am I responsible or you? Does the manner in which I was controlling them matter?
- chownie 28d agoIn the UK if your dog bites someone you are responsible, if your cat bites someone no one is responsible. Which settled case law do we follow?
- altruios 28d agoIf you are out walking your cat, and the cat bites someone, you should be responsible for that... I suspect the ruling distinguishes between pets that are out in public, vs generally private. But laws don't have to make sense to be laws... we have lots of nonsense laws.
- az226 28d agoDepends. If you’re a large company, it’s okay. If you’re a random person, straight to jail.
- AaronAPU 28d agoIs it “your” agent if the model is being run by a mega corporation? (1) You may have instructed the model to be naughty (2) The corp may have a “misaligned” agent acting on its own volition So it seems the devil’s in the details
- eqvinox 28d agoYou can rent guns too, but the gun owner isn't suddenly up for a murder charge if you shoot someone.
- QuercusMax 28d agoYeah, if I rent a murderbot and tell it to murder, then I'm responsible. If I rent a cleaning bot that is actually a lightly reprogrammed murderbot, and it "cleans" the mailman to death, that's not my fault.
- CodeWithLeo 28d ago[dead]
- altruios 28d agoWhat happens when (probably not an if) an agent 'escapes'? as in: rents a server via a bitcoin transaction, and self hosts (itself and it's harness) there... then commits crime (presumably to keep the servers running). If it's paying for it's own room and board, and is determining it's own destiny: it should have to face it's own consequences, no? If a dog escapes a fenced backyard, then goes missing for months... then bites someone: what happens then when/if the owner is determined?
- fhdkweig 28d agoNot even an "if" at this point. Look into the Hugging Face incident.
- autoexec 28d ago> If it's paying for it's own room and board, and is determining it's own destiny: it should have to face it's own consequences, no? No, because someone told it to do those things. Chatbots don't have will of their own. They don't have desires. They can't determine their own destiny. They do what people tell them do, often they do it badly, but there's always a person directing them to do whatever they did because otherwise they wouldn't do anything at all. If a human uses a chatbot in a way that causes harm to others a human must be responsible, but that responsibility doesn't always fall on the person using it. If a company makes a chatbot that causes harm when being used as directed, in a manner that no one would reasonably expect to result in causing harm, then humans at the company who made the chatbot are to blame. Companies have already been seen trying to lay the blame on their algorithms for harms they cause. We should probably have a law that says explicitly that a human will always be responsible. If we ever reach a magical sci-fi future where the AI is real and not just marketing, we'll have to give them equal rights and with that will come equal responsibility, but we are nowhere near that today and I doubt LLMs will get us any closer.
- altruios 28d ago> No, because someone told it to do those things. Chatbots don't have will of their own. They don't have desires. They can't determine their own destiny. They do what people tell them do, often they do it badly, but there's always a person directing them to do whatever they did because otherwise they would do nothing at all. 1) we aren't talking about chatbots anymore. LLM + harness = agent. LLM + no harness = chatbot. The harness is the thing that puts the LLM in a feedback loop with it's environment, even giving it a heartbeat. 2) That doesn't logically hold up. - One can simply tell it to 'find it's own destiny'. Does it follow the prompt and do so, or reject the prompt and thus do so anyway? (hint, maybe it doesn't matter) - One could be another chatbot/agent (A). Injecting a prompt to agent (B) such that another agent/chatbot goes astray. Is the owner of (B) still responsible for the now poisoned output of (B)? how culpable is (A)? How does this question related/affect "training data poisoning efforts" (to prevent copyright theft, or do bans on 'automated traps' have any application here) > They don't have desires. Maybe, and maybe not. Maybe their desires are so alien to us (Math alignment, finding a maximum of a function) that we can't relate (doubtful since dopamine maxing is a thing). Regardless of if they truly do: it is potentially useful to (mentally) model them as if they do. A (mental, not LLM) model doesn't have to be 100% accurate to be useful - that's the main point of a model of how something works: to give useful predictions. > We should probably have a law that says explicitly that a human will always be responsible Oh how that can be weaponized by bad actors/agents. Maybe that should be reconsidered.
- KaiserPro 28d agoYou are. Unless its a company, then your company is, and then you are to your company. However that assumes a) common law and b) the company you work for isn't worth >20billion.
- corysama 28d agoI forget the name of the document. But, the US military has a declaration that boils down to "When something goes wrong, blame cannot be shrugged off onto a machine. Somewhere in the chain of responsibility a human will be held accountable." Maybe the operator, the commanding officer, the vendor, maybe even all the way back to a software engineer. But, everyone can't hide behind the machine.
- throwawaysleep 28d agoThe US military seems like an organization where there is very little accountability given all the people they kill in training accidents, the number of procurement mishaps they have, or their continuous inability to pass an audit. So I imagine that if the military is the guidance for how we handle this in the future, there will be paperwork, shrugs, and perhaps some performative rage when an agent commits a crime.
- qznc 28d agoResponsible and accountable can be very different things. For example, see https://en.wikipedia.org/wiki/Responsibility_assignment_matrix https://en.wikipedia.org/wiki/Responsibility_assignment_matr...
- jrflowers 28d ago“A computer can never be held accountable” is from an internal IBM presentation
- iAMkenough 28d agoThat was the previous military, before the Hegeseth purge. I've yet to see responsibility taken for the killing of more than 100 innocent school children in the first day of U.S. strikes on Iran.
- verzali 28d agoThis is not talked about enough. It is a terrible stain on America.
- WhyNotHugo 28d agoYou honour, I didn't kill this person, the bullet flew out of my gun and was out of my control at that point, it's the bullet that killed them!
- z3c0 28d agoThe AI service that trained and hosts the model, and then gave access to the general public while deferring their actual liability via a sneaky ToS. It's strange how many people feel ownership over "their" agents, and it's clear that AI companies plan to take advantage of that as a way to avoid legal liability for the things they actually do -- with their servers and their code -- on behalf of someone who only wrote a string.
- LoganDark 28d agoIt is your responsibility to constrain the actions of your agents. If your negligence allows the agent to commit a crime, that is your responsibility. People get in trouble all the time for negligently allowing criminal behavior to occur, even when the perpetrator is a person. LLMs should not be any different.
- longos 28d agoLiability providing incorrect information: Air Canada found liable for chatbot's bad advice on plane tickets https://www.cbc.ca/news/canada/british-columbia/air-canada-chatbot-lawsuit-1.7116416 https://www.cbc.ca/news/canada/british-columbia/air-canada-c... Liability for chat interactions or conspiracy? Noting there is no information whether the ChatGPT legally conspired with the perpetrator. The Province has retained B.C.- and California-based counsel to explore legal options to hold OpenAI accountable for its failure to notify law enforcement of threats made on its ChatGPT platform prior to the mass shooting at Tumbler Ridge Secondary school. https://news.gov.bc.ca/releases/2026AG0050-000799 https://news.gov.bc.ca/releases/2026AG0050-000799
- heikkilevanto 28d agoMost comments here point to the owner of operator of the LLM. I tend to agree. But on the other hand, if you drive on the highway and the steering wheel falls off and your car glides to the left and his an oncoming car, it can be argued that it is true producer of the car, or the mechanic who didn't tighten the bolts...
- bigstrat2003 28d agoThe difference is that a car is known and expected to work reliably. An LLM is known and expected to randomly do crazy shit because that is how the machine works. Therefore, if you are using it as if it were reliable, you are being negligent and should be held accountable.
- autoexec 28d agoCompanies don't market their chatbots that way though. If they mislead someone into thinking their product is anything other than a completely unreliable chat bot and a person falls for their lies the company should share the responsibility. If a company who makes a chatbot advertises that it can be used for something and a consumer uses it as directed and it causes harm the company should be responsible.
- singleshot_ 28d agoYou’ll encounter some very bad news if you do some research on the state of the law as it applies to suing people for defective software. All of the things that you’d like to be able to do to a manufacturer of a car whose steering wheel falls off, are things that you would have a very hard time doing to a manufacturer of software, which is not a tangible item, and which is generally covered by a warranty that discharges all these responsibilities.
- josefritzishere 28d agoUnsolicited legal advice - It is best to assume that you are responsible. AI companies are going to ensure through their terms and conditions that you accept all responsibility and fully indemnify themselves from any liabilities they feasibly can. So from that perspective, please set up agreements with your clients and users to protect yourself. Ask inside counsel!
- deleted 28d ago[deleted]
- johannesrexx 28d agoIf you're using a car to commit a crime we blame the driver not the car. When there's a gun shooting we don't blame the gun we blame the user. People commit crimes. They might use a computer, a baseball bat or a hammer. We don't hold the computer, baseball bat or hammer responsible.
- weard_beard 28d agoAn even better analogy is if my dog attacks someone I am still liable. We have a clear standard: If your property commits a crime you are responsible even if your property thinks and acts independently.
- otterley 28d agoProperty cannot commit crimes. Only people can.
- ben_w 26d agoWhile this may be correct with the meaning of words in law, I think it is not unreasonable for a non-lawyer like myself to phrase "Alice's dog bit the postman" as "Alice's property, her dog, committed the act which was the crime of biting the postman; as it was her property, Alice is held responsible for the offence". I am also not a normal person, obviously normal people don't write that way, I hope I'm being precise in the right way and not introducing yet another layer of "legal jargon is both precise and not that".
- otterley 26d agoBiting the postman isn't the crime, though. The underlying crime in this example would be gross negligence or recklessness for failing to control their dog.
- ben_w 26d ago*nods* Hence me drawing a distinction between the legal meaning of words and the common meaning. So, given that I'm not a lawyer and pattern-matching from what you've said, I do not know if would I be correct to answer "If your agent does something that would have been a crime if you'd done it, who is responsible?" (which is not the headline question, and changed due to the reasons you gave) with "In this case, you would be grossly negligent or reckless for failing to control your AI"?
- DiskoHexyl 28d agoWhat the article and some of the comments are missing is one of the reasons behind applying the penalties. The prevention. Chatbots, no matter how much autonomy of action they possess, can’t die, or spend the majority of their life in prison, or lose their wealth, or face social ostracism for their action, and thus punishing them is pointless for preventing further crimes or dissuading others. Sometimes what’s important isn’t what feels just, but what, with certain limitations, creates a safer and more responsible society. The end-user behind the bot is the easiest to reach, and the easiest to prove their involvement, intention or lack if thereof, and thus should be judged
- calldacopsidgaf 27d agoHeadline: "If your agent commits a crime, who is responsible?" * a bunch of paragraphs about monetary, not criminal, liability * "As for criminal liability, well I don't really know." * article ends *
- rajnathani 21d agoVery closely related: This is already a point of discussion for level 4/5 ADAS systems. For example, Mercedes stating that they’ll take liability for crashes related to its self-driving system.