7 ms·

I don't know, it feels like "reading a claims summary and classifying the type of claim" should be the bread and butter LLM use case? Not that executives are bl

I don't know, it feels like "reading a claims summary and classifying the type of claim" should be the bread and butter LLM use case? Not that executives are blameless for pushing AI everywhere, but we should also be able to "blame" AI for doing poorly at a task it's supposed to be good at.

fluoridation 14d ago
[flagged]
dpoloncsak 14d ago
A computer can never be held accountable, therefore a computer must never make a management decision. You can't 'blame' the AI, as it can't be held accountable.
- voncheese 14d ago
  The interesting thing about AI v. other disruptive technologies (like the internet, steam engine, etc), is that AI has been personified both by the manufacturers of it (Anthropic being the most blatant by using a human name for its product) and users. That makes it way more likely that people blame AI (doesn't make it right by any means, but does mean that the actual blame gets diffused even more).
  - spwa4 13d ago
    How is that a problem? That makes it far worse for companies, after all. If an employee of a company makes a mistake then the consequences are: 1) employee was willingly sabotaging the company: employee gets jail time plus fine, employer is on the hook financially 2) employee was involved either in accident or just made a stupid decision: employer is on the hook financially So if that's the case, then employers are financially responsible for everything the AI they use says. That's the rule for employees. Note: this is different from, say if a company uses a car and it just starts by itself and drives into the next door warehouse setting it on fire (ie. some sort of designed-in decision/issue) : in that case the maker of the car is responsible, both financially and criminally, even if an employee or even the boss/owner/everyone at the company were involved. Of course, guaranteed that governments will change this the first time a court makes this obvious connection. Why do I think that? Well, for example, the UK government changed the rule when it turned out the post-office's written software (written by contractors, under post office and thus government responsibility and accountability) decided to cause incredible damage by blaming innocent people for stealing. Then, a special law was voted that in that specific case it didn't apply, and the government department, nor any of it's employees, were responsible. Just like, for example, Pennsylvania changed the rules for unwittingly-but-directly aiding criminal activity when it turned out half the Pennsylvania government was complicit in the kids-for-cash scandal (because in that case, normally, you wouldn't be criminally liable, but you WOULD be liable for any financial damage you do)
    - voncheese 9d ago
      Regarding "how is that a problem" it has nothing to do with the financial obligations (I agree with your take that the company is on the hook either way). It's about being able to learn and improve. In order to learn and improve you have to know what went wrong, with AI being personified and pulled into the pool of entities that may be at fault, it makes it harder to figure out what actually went wrong and learn. Not impossible to figure out, just harder.
- dsign 14d ago
  Though your statements are generally true today, I wonder if I can put a bet somewhere about how long it takes until somebody "fixes that problem."
- deleted 14d ago
  [deleted]
- antasvara 14d ago
  >You can't 'blame' the AI, as it can't be held accountable. Seems like more of a semantic distinction IMO. Yes, I can't technically "blame" the AI because it's not accountable. But what word would you suggest for "I had [X technology] handle [Y task] and it failed to perform that task?" I can't "blame" my router if I lose internet and it prevents me from jumping on a Zoom call, but it's also true that the router failed to do the thing it was supposed to do.
  - dpoloncsak 14d ago
    Do you blame the hammer for bending the nail, or the person driving it? AI is nothing more than a tool being used by people. When my internet goes down, I blame either my ISP for having an outage, or the manufacturer of my router (assuming it's a router issue? Haven't had that happen personally but I'm certain it exists) (Or i did something dumb with my ufw again but that's on me) If you create an autonomous system and it fails, blame should be on you, imo. The word is still 'blame', it just needs to be applied correctly.
    - layer8 14d ago
      Humans are autonomous systems created by other humans. Yet, at some point, we stop blaming parents for the actions of their children. LLMs aren’t there yet, but they might get there. My point is that your descriptions fail to capture when or why exactly the blame would shift.
      - away0g 14d ago
        llms have the ability to act without being affected by the actions, consequences or successes. what it means to blame has to be defined.
WarmWash 14d ago
Not that it is necessarily the case here, but when execs go "AI shopping" they have absolutely _zero_ idea that there are basically 3-4 SOTA models, thousands of smaller models, and then an uncountable number of wrappers on whatever underlying model. The AI landscape that is totally familiar to us is covered in shroud for them. So they Google "Insurance claim AI tool", land on a vibecoded SaaS that is just a wrapper on a pocket Chinese model spun as "your next insurance pro", and then are getting the whole department on some lone 19 yr olds weekend project.
labcomputer 14d ago
Part of the problem is the distribution of images (and text) you get from claims is not the same as what the model was trained on. A classic problem in ML. Another part of the problem is that a model not specifically fine-tuned to make a total loss determination won't know the relevant factors, nor how an insurance company's concept of a total loss differs from the public's. And still another part of the problem is that most total loss claims aren't what you, dear reader, are imagining: They are very rarely "the car is a thin pancake after being crushed by a meteor". The much, much more common scenario is: "50% of the body panels sustained at least paint damage, both headlight modules need replacement, and the front wheels look funny. Given that the vehicle has an MSRP of $FOO, $BAR miles, no prior collision history on carfax, and is a popular color, is it cheaper to repair or total the vehicle?" Of course, the model can turn over the hard cases to a human adjuster... but then what are we doing here? It only takes 10 seconds for the human adjuster to handle the "crushed by a meteor" case also. Source: Listening to my SIL rant about being asked to stop training bespoke total loss models and just send it by 1-shotting a commercial LLM.

fluoridation 14d ago

[flagged]

dpoloncsak 14d ago

A computer can never be held accountable, therefore a computer must never make a management decision. You can't 'blame' the AI, as it can't be held accountable.

voncheese 14d ago
The interesting thing about AI v. other disruptive technologies (like the internet, steam engine, etc), is that AI has been personified both by the manufacturers of it (Anthropic being the most blatant by using a human name for its product) and users. That makes it way more likely that people blame AI (doesn't make it right by any means, but does mean that the actual blame gets diffused even more).
- spwa4 13d ago
  How is that a problem? That makes it far worse for companies, after all. If an employee of a company makes a mistake then the consequences are: 1) employee was willingly sabotaging the company: employee gets jail time plus fine, employer is on the hook financially 2) employee was involved either in accident or just made a stupid decision: employer is on the hook financially So if that's the case, then employers are financially responsible for everything the AI they use says. That's the rule for employees. Note: this is different from, say if a company uses a car and it just starts by itself and drives into the next door warehouse setting it on fire (ie. some sort of designed-in decision/issue) : in that case the maker of the car is responsible, both financially and criminally, even if an employee or even the boss/owner/everyone at the company were involved. Of course, guaranteed that governments will change this the first time a court makes this obvious connection. Why do I think that? Well, for example, the UK government changed the rule when it turned out the post-office's written software (written by contractors, under post office and thus government responsibility and accountability) decided to cause incredible damage by blaming innocent people for stealing. Then, a special law was voted that in that specific case it didn't apply, and the government department, nor any of it's employees, were responsible. Just like, for example, Pennsylvania changed the rules for unwittingly-but-directly aiding criminal activity when it turned out half the Pennsylvania government was complicit in the kids-for-cash scandal (because in that case, normally, you wouldn't be criminally liable, but you WOULD be liable for any financial damage you do)
  - voncheese 9d ago
    Regarding "how is that a problem" it has nothing to do with the financial obligations (I agree with your take that the company is on the hook either way). It's about being able to learn and improve. In order to learn and improve you have to know what went wrong, with AI being personified and pulled into the pool of entities that may be at fault, it makes it harder to figure out what actually went wrong and learn. Not impossible to figure out, just harder.
dsign 14d ago
Though your statements are generally true today, I wonder if I can put a bet somewhere about how long it takes until somebody "fixes that problem."
deleted 14d ago
[deleted]
antasvara 14d ago
>You can't 'blame' the AI, as it can't be held accountable. Seems like more of a semantic distinction IMO. Yes, I can't technically "blame" the AI because it's not accountable. But what word would you suggest for "I had [X technology] handle [Y task] and it failed to perform that task?" I can't "blame" my router if I lose internet and it prevents me from jumping on a Zoom call, but it's also true that the router failed to do the thing it was supposed to do.
- dpoloncsak 14d ago
  Do you blame the hammer for bending the nail, or the person driving it? AI is nothing more than a tool being used by people. When my internet goes down, I blame either my ISP for having an outage, or the manufacturer of my router (assuming it's a router issue? Haven't had that happen personally but I'm certain it exists) (Or i did something dumb with my ufw again but that's on me) If you create an autonomous system and it fails, blame should be on you, imo. The word is still 'blame', it just needs to be applied correctly.
  - layer8 14d ago
    Humans are autonomous systems created by other humans. Yet, at some point, we stop blaming parents for the actions of their children. LLMs aren’t there yet, but they might get there. My point is that your descriptions fail to capture when or why exactly the blame would shift.
    - away0g 14d ago
      llms have the ability to act without being affected by the actions, consequences or successes. what it means to blame has to be defined.

WarmWash 14d ago

Not that it is necessarily the case here, but when execs go "AI shopping" they have absolutely _zero_ idea that there are basically 3-4 SOTA models, thousands of smaller models, and then an uncountable number of wrappers on whatever underlying model. The AI landscape that is totally familiar to us is covered in shroud for them. So they Google "Insurance claim AI tool", land on a vibecoded SaaS that is just a wrapper on a pocket Chinese model spun as "your next insurance pro", and then are getting the whole department on some lone 19 yr olds weekend project.

labcomputer 14d ago

Part of the problem is the distribution of images (and text) you get from claims is not the same as what the model was trained on. A classic problem in ML. Another part of the problem is that a model not specifically fine-tuned to make a total loss determination won't know the relevant factors, nor how an insurance company's concept of a total loss differs from the public's. And still another part of the problem is that most total loss claims aren't what you, dear reader, are imagining: They are very rarely "the car is a thin pancake after being crushed by a meteor". The much, much more common scenario is: "50% of the body panels sustained at least paint damage, both headlight modules need replacement, and the front wheels look funny. Given that the vehicle has an MSRP of $FOO, $BAR miles, no prior collision history on carfax, and is a popular color, is it cheaper to repair or total the vehicle?" Of course, the model can turn over the hard cases to a human adjuster... but then what are we doing here? It only takes 10 seconds for the human adjuster to handle the "crushed by a meteor" case also. Source: Listening to my SIL rant about being asked to stop training bespoke total loss models and just send it by 1-shotting a commercial LLM.