10 ms·

"In an argument that appeared to flabbergast a small claims adjudicator in British Columbia, the airline attempted to distance itself from its own chatbot's bad

by ooboe 3y ago

"In an argument that appeared to flabbergast a small claims adjudicator in British Columbia, the airline attempted to distance itself from its own chatbot's bad advice by claiming the online tool was "a separate legal entity that is responsible for its own actions." "This is a remarkable submission," Civil Resolution Tribunal (CRT) member Christopher Rivers wrote."" From https://www.cbc.ca/news/canada/british-columbia/air-canada-chatbot-lawsuit-1.7116416 https://www.cbc.ca/news/canada/british-columbia/air-canada-c...

onlyrealcuzzo 3y ago
How is this different from me getting one of my friends to work at Air Canada and promise me a billion dollars to cancel my flight? Will Air Canada be legal for my friend going against company policy?
- nrmitchi 3y ago
  What you are describing is 1) fraud, 2) conspiracy, and 3) not a policy that a reasonable person would take at face value. It is very different than if an employee were to, in writing, make a statement that a reasonable person would find reasonable.
- weego 3y ago
  Likely because the claim was considered to be within the reasonable expectations of real policy.
- vundercind 3y ago
  The computer only does what they told it to. What they told it to do was to behave very unpredictably. They shouldn’t have done that.
  - philipswood 3y ago
    Not these ones... These ones do what they "learned" from a lot of input data using a process that is us mimicking how we think brains could maybe function (kinda/sort off with a few unbiological "improvements").
    - vundercind 3y ago
      Yes, these ones. Somebody told the computer to do all those things you just wrote.
      - philipswood 3y ago
        Maybe this makes the point better: Say your webserver isn't scaling to more than 500 concurrent users. When you add more load, connections start dropping. Is it because someone programmed a max_number_of_concurrent_users variable and a throttleExtraAboveThresholdRequests() function? No. Yes, humans built the entire stack of the system. Yes every part of it was "programmed", but no this behaviour wasn't programmed intentionally, it is an emergent property arising from system constraints. Maybe the database connection pool is maxed out and the connections are saturating. Maybe some database configuration setting is too small or the server has too few file handles - whatever. Whatever the root cause (even though that cause incidentally was implemented by a human if you trace the causal chain back far enough) this behaviour is an almost incidental unintended side effect of that. A machine learning system is like that, but more so. An LLM, say, is "parsing" language in some sense, but ascribing what it is doing to human design is pretty indirect. In a way you typing words at me has in some way been "programmed" into you by every language interaction mankind has had with you. I guess you could see it that way, but I don't think it's a particularly useful point of view. In the same way an LLM has been in directly "programmed" via it's model architecture, training algorithm and training data, but we are nowhere near the understanding of the process to be able to consider this "programming" it yet.
        vundercind 3y ago
        This is different from a bug or hitting an unknown limitation—the selling point of this was “it makes shit up” and they went “yeah, cool, let’s have it speak for us”. Its behavior incorporates randomness and is unpredictable and hard to keep within bounds on purpose and they decided to tell a computer to follow that unpredictable instruction set and place it in a position of speaking for the company, without a human in between. They shouldn’t have done that if they didn’t want to end up in this sort of position.
- BadHumans 3y ago
  Chatbots aren't people and people are actually separate legal entities responsible for their own actions.
  - kube-system 3y ago
    People working for companies are sometimes separate legal entities responsible for their own actions, and sometimes they act on behalf of the company they work for and the company is responsible for their actions. It depends.
    - BadHumans 3y ago
      A chatbot(computer) cannot be responsible for their own actions so the only half of the coin you have left is "the company is responsible for their actions."
      - kube-system 3y ago
        Computers are inanimate objects and are not recognized as legal entities. The legal system recognizes that people, or groups of people, are subject to legal authority. This is a story about a piece of software Air Canada implemented which resulted in them posting erroneous information on their website.
- carlosjobim 3y ago
  Law and justice is not like a computer program that you can exploit and control without limits by being a hacker. If the chatbot told them that they'd get a billion dollars, the courts would not hold Air Canada responsible for it, just as if a programmer put a decimal wrong and prices became obviously wrong. In this case, the chat bot gave a policy within reason and the court awarded the passenger what the bot had promised, which is a completely correct judgement.
  - Delumine 3y ago
    This argument seems overly dramatic and distorted. Yes, in an outrageous situation like a billion-dollar mishap, most people would know something isn't right. But for a policy that appears legitimate, especially when it's replacing a human customer service rep, it's not that obvious. In these cases, Air Canada should definitely be held accountable.
    - carlosjobim 3y ago
      Yes, that's exactly what I'm saying as well. Especially since they had already taken the customer's money.
- BiteCode_dev 3y ago
  Your friend is not trained by Air Canada. The bot is Air Canada property. If they decide it is reliable enough to be put in front of the customer, they must accept all the consequences: the benefits like having to hire less, and the cons, which is that they have to make it work correctly. Otherwise, woopsy, we made our AI handle our accounting and it cheated, sorry IRS. That won't fly.
- deleted 3y ago
  [deleted]
- willcipriano 3y ago
  You didn't get your friend to do it, an employee just decided to. There is no conspiracy.
- vkou 3y ago
  >How is this different from me getting one of my friends to work at Air Canada and promise me a billion dollars to cancel my flight? There is a common misconception about law that software engineers have. Code is not law. Law is not code. Just because something that looks like a function exists, you can't just plug in any inputs and expect it to have a consistent outcome. The difference between these two cases is that even if a chat bot promised that, the judge would throw it out, because it's not reasonable. Also, the firm would have a great case against at least the CS rep for this collusion. If your friend of a CS agent promised you a bereavement refund (As the chatbot did), even though it went against company policy, you'd have good odds of winning that case. Because the judge would find it reasonable of you to believe and expect that after speaking to a CS rep, that such a policy would actually get honored. (And the worst that would happen to the CS rep would be termination.)
- nneonneo 3y ago
  Because that would not be reasonable, and nobody would be surprised if Air Canada reneged on that. See, for instance, Leonard vs. PepsiCo. If your friend promised you something reasonable in the course of carrying out their duties, and you honestly believed them, I think that would be legal and enforceable just as this case suggests.
- thsksbd 3y ago
  That's fraud because you're in cahoots with your friend. If a random AC employee gave you a free flight, on the other hand, you'd be entitled to it. Anyway, the chat bot has no agency except that given to it by AC; unlike a human employee, therefore, its actions are 100% AC actions. I don't see how this is controversial? Why do people think that laws no longer apply when fancy high-tech pixie dust is sprinkled?
  - baggy_trough 3y ago
    A random AC employee drinks too much and says "You are entitled to free flights for the rest of your life." Is Air Canada liable?
    - wredue 3y ago
      Since when are contracts enforceable when one party is drunk?
      - baggy_trough 3y ago
        A random AC employee who is having a bad day and hates his employer says "You are entitled to free flights for the rest of your life." Is Air Canada liable?
        kube-system 3y ago
        Valid contracts usually require consideration
        acdha 3y ago
        No, because no reasonable person would think that they had the authority to authorize that. Remember, the legal system is not computer code - judges look at things like intent and plausibility.
        DiggyJohnson 3y ago
        No, because that's not "reasonable". My dad jokes that he's made a career off of determining what is "reasonable" and what isn't, and he's a contract attorney. If you were standing at the customer service desk, and instead they said: "sorry about the delay, your next two flights are free", then all of a sudden this is "reasonable".
- spamizbad 3y ago
  Your example is significantly different. The chatbot instructed the passenger to pay full price for a ticket but stated they could get a refund later. That refund policy was a hallucination. The victim her just walked away with a discounted ticket as promised not a billion dollars.
- deleted 3y ago
  [deleted]
- hiddencost 3y ago
  Weird straw man... So replacing all their customer support staff with AI that misleads customers is OK? That's pants on head insane, so why spend time trying to justify it.
- oliwary 3y ago
  No, it is more similar to Air Canada hiring a monkey to push buttons to handle customer complaints. In that case, the company knows (or should know) that the given information may be wrong, but accepts the risk.
- Fnoord 3y ago
  AI has auth from higher up to be used 1:1 as truth. Your friend does not have auth to promise you 1 billion in their employers name.
- dataflow 3y ago
  > How is this different from me getting one of my friends to work at Air Canada One major difference is the AI wasn't your friend, another is that you didn't get it hired at Air Canada, another is that the promise wasn't $1B, etc...
- TheCoelacanth 3y ago
  No, if you conspire with your friend to get them to tell you an incorrect policy, then you have no reasonable expectation that what they tell you is the real policy. If you are promised a billion dollars even without a pre-existing relationship with the agent, you have no reasonable expectation that what they are promising is the real policy because it's an unbelievably large amount. If you are promised something reasonable by an agent of the company who you are not conspiring with, then the company is bound to follow through on the promise because you do have a reasonable expectation that what they are telling you is the real policy.
- eirikbakke 3y ago
  The legal concept is called "Apparent authority". The test is whether "a reasonable third party would understand that an agent had authority to act". ("Chatbot says you can submit a form within 90 days to get a retroactive bereavement discount" sounds perfectly reasonable, so the doctrine applies.) https://en.wikipedia.org/wiki/Apparent_authority https://en.wikipedia.org/wiki/Apparent_authority
jasonjayr 3y ago
IANAL, but it's astounding they took that as their defense, rather than pointing to a line (I hope?) in their ToS that says "This agreement is the complete terms of service, and cannot be amended or changed by any agent or representative of the company except by ... (some very specific process the bot can't follow)". I've seen this mentioned in several ToSs, I expect it to be standard boilerplate at this point ...
- JCM9 3y ago
  Courts often rule that you can’t use ToS to overcome common sense. ToS are not a get out of jail free card if your company just does stupid things.
- AlexandrB 3y ago
  How is that enforceable? In many cases this is carte blanche for company representatives to lie to you. No one is going to read the ToS and cross reference it with what they're being told in real time. Moreover, if a customer was familiar with the ToS they would not be asking questions like this of a chatbot. The entire idea of having a clause like this while also running a "help" chatbot that can contradict it seems like bad faith dealing.
- dataflow 3y ago
  How do those clauses actually work? If a rep does something nice for you (like give you something for free), could the airline say it never agreed to that in writing or whatever and demand it back? How are you supposed to know if a rep has authority to enter into an agreement with you over random matters? But, to your question, my guess is that would basically be telling people not to avoid their chatbot, which they don't want to do.
  - paxys 3y ago
    It's more to shield them from cases like a rep gifting you free flights for life.
    - dataflow 3y ago
deleted 3y ago
[deleted]
animex 3y ago
Straight out of I, Robot.
skywhopper 3y ago
The claim is so outrageous that I wish there were a way (I assume there probably isn't) for the company or the lawyers to have been sanctioned outside what the plaintiff was asking for.

by ooboe 3y ago

onlyrealcuzzo 3y ago

How is this different from me getting one of my friends to work at Air Canada and promise me a billion dollars to cancel my flight? Will Air Canada be legal for my friend going against company policy?

nrmitchi 3y ago
What you are describing is 1) fraud, 2) conspiracy, and 3) not a policy that a reasonable person would take at face value. It is very different than if an employee were to, in writing, make a statement that a reasonable person would find reasonable.
weego 3y ago
Likely because the claim was considered to be within the reasonable expectations of real policy.
vundercind 3y ago
The computer only does what they told it to. What they told it to do was to behave very unpredictably. They shouldn’t have done that.
- philipswood 3y ago
  Not these ones... These ones do what they "learned" from a lot of input data using a process that is us mimicking how we think brains could maybe function (kinda/sort off with a few unbiological "improvements").
  - vundercind 3y ago
    Yes, these ones. Somebody told the computer to do all those things you just wrote.
    - philipswood 3y ago
      Maybe this makes the point better: Say your webserver isn't scaling to more than 500 concurrent users. When you add more load, connections start dropping. Is it because someone programmed a max_number_of_concurrent_users variable and a throttleExtraAboveThresholdRequests() function? No. Yes, humans built the entire stack of the system. Yes every part of it was "programmed", but no this behaviour wasn't programmed intentionally, it is an emergent property arising from system constraints. Maybe the database connection pool is maxed out and the connections are saturating. Maybe some database configuration setting is too small or the server has too few file handles - whatever. Whatever the root cause (even though that cause incidentally was implemented by a human if you trace the causal chain back far enough) this behaviour is an almost incidental unintended side effect of that. A machine learning system is like that, but more so. An LLM, say, is "parsing" language in some sense, but ascribing what it is doing to human design is pretty indirect. In a way you typing words at me has in some way been "programmed" into you by every language interaction mankind has had with you. I guess you could see it that way, but I don't think it's a particularly useful point of view. In the same way an LLM has been in directly "programmed" via it's model architecture, training algorithm and training data, but we are nowhere near the understanding of the process to be able to consider this "programming" it yet.
      - vundercind 3y ago
        This is different from a bug or hitting an unknown limitation—the selling point of this was “it makes shit up” and they went “yeah, cool, let’s have it speak for us”. Its behavior incorporates randomness and is unpredictable and hard to keep within bounds on purpose and they decided to tell a computer to follow that unpredictable instruction set and place it in a position of speaking for the company, without a human in between. They shouldn’t have done that if they didn’t want to end up in this sort of position.
BadHumans 3y ago
Chatbots aren't people and people are actually separate legal entities responsible for their own actions.
- kube-system 3y ago
  People working for companies are sometimes separate legal entities responsible for their own actions, and sometimes they act on behalf of the company they work for and the company is responsible for their actions. It depends.
  - BadHumans 3y ago
    A chatbot(computer) cannot be responsible for their own actions so the only half of the coin you have left is "the company is responsible for their actions."
    - kube-system 3y ago
      Computers are inanimate objects and are not recognized as legal entities. The legal system recognizes that people, or groups of people, are subject to legal authority. This is a story about a piece of software Air Canada implemented which resulted in them posting erroneous information on their website.
carlosjobim 3y ago
Law and justice is not like a computer program that you can exploit and control without limits by being a hacker. If the chatbot told them that they'd get a billion dollars, the courts would not hold Air Canada responsible for it, just as if a programmer put a decimal wrong and prices became obviously wrong. In this case, the chat bot gave a policy within reason and the court awarded the passenger what the bot had promised, which is a completely correct judgement.
- Delumine 3y ago
  This argument seems overly dramatic and distorted. Yes, in an outrageous situation like a billion-dollar mishap, most people would know something isn't right. But for a policy that appears legitimate, especially when it's replacing a human customer service rep, it's not that obvious. In these cases, Air Canada should definitely be held accountable.
  - carlosjobim 3y ago
    Yes, that's exactly what I'm saying as well. Especially since they had already taken the customer's money.
BiteCode_dev 3y ago
Your friend is not trained by Air Canada. The bot is Air Canada property. If they decide it is reliable enough to be put in front of the customer, they must accept all the consequences: the benefits like having to hire less, and the cons, which is that they have to make it work correctly. Otherwise, woopsy, we made our AI handle our accounting and it cheated, sorry IRS. That won't fly.
deleted 3y ago
[deleted]
willcipriano 3y ago
You didn't get your friend to do it, an employee just decided to. There is no conspiracy.
vkou 3y ago
>How is this different from me getting one of my friends to work at Air Canada and promise me a billion dollars to cancel my flight? There is a common misconception about law that software engineers have. Code is not law. Law is not code. Just because something that looks like a function exists, you can't just plug in any inputs and expect it to have a consistent outcome. The difference between these two cases is that even if a chat bot promised that, the judge would throw it out, because it's not reasonable. Also, the firm would have a great case against at least the CS rep for this collusion. If your friend of a CS agent promised you a bereavement refund (As the chatbot did), even though it went against company policy, you'd have good odds of winning that case. Because the judge would find it reasonable of you to believe and expect that after speaking to a CS rep, that such a policy would actually get honored. (And the worst that would happen to the CS rep would be termination.)
nneonneo 3y ago
Because that would not be reasonable, and nobody would be surprised if Air Canada reneged on that. See, for instance, Leonard vs. PepsiCo. If your friend promised you something reasonable in the course of carrying out their duties, and you honestly believed them, I think that would be legal and enforceable just as this case suggests.
thsksbd 3y ago
That's fraud because you're in cahoots with your friend. If a random AC employee gave you a free flight, on the other hand, you'd be entitled to it. Anyway, the chat bot has no agency except that given to it by AC; unlike a human employee, therefore, its actions are 100% AC actions. I don't see how this is controversial? Why do people think that laws no longer apply when fancy high-tech pixie dust is sprinkled?
- baggy_trough 3y ago
  A random AC employee drinks too much and says "You are entitled to free flights for the rest of your life." Is Air Canada liable?
  - wredue 3y ago
    Since when are contracts enforceable when one party is drunk?
    - baggy_trough 3y ago
      A random AC employee who is having a bad day and hates his employer says "You are entitled to free flights for the rest of your life." Is Air Canada liable?
      - kube-system 3y ago
        Valid contracts usually require consideration
      - acdha 3y ago
        No, because no reasonable person would think that they had the authority to authorize that. Remember, the legal system is not computer code - judges look at things like intent and plausibility.
      - DiggyJohnson 3y ago
        No, because that's not "reasonable". My dad jokes that he's made a career off of determining what is "reasonable" and what isn't, and he's a contract attorney. If you were standing at the customer service desk, and instead they said: "sorry about the delay, your next two flights are free", then all of a sudden this is "reasonable".
spamizbad 3y ago
Your example is significantly different. The chatbot instructed the passenger to pay full price for a ticket but stated they could get a refund later. That refund policy was a hallucination. The victim her just walked away with a discounted ticket as promised not a billion dollars.
deleted 3y ago
[deleted]
hiddencost 3y ago
Weird straw man... So replacing all their customer support staff with AI that misleads customers is OK? That's pants on head insane, so why spend time trying to justify it.
oliwary 3y ago
No, it is more similar to Air Canada hiring a monkey to push buttons to handle customer complaints. In that case, the company knows (or should know) that the given information may be wrong, but accepts the risk.
Fnoord 3y ago
AI has auth from higher up to be used 1:1 as truth. Your friend does not have auth to promise you 1 billion in their employers name.
dataflow 3y ago
> How is this different from me getting one of my friends to work at Air Canada One major difference is the AI wasn't your friend, another is that you didn't get it hired at Air Canada, another is that the promise wasn't $1B, etc...
TheCoelacanth 3y ago
No, if you conspire with your friend to get them to tell you an incorrect policy, then you have no reasonable expectation that what they tell you is the real policy. If you are promised a billion dollars even without a pre-existing relationship with the agent, you have no reasonable expectation that what they are promising is the real policy because it's an unbelievably large amount. If you are promised something reasonable by an agent of the company who you are not conspiring with, then the company is bound to follow through on the promise because you do have a reasonable expectation that what they are telling you is the real policy.
eirikbakke 3y ago
The legal concept is called "Apparent authority". The test is whether "a reasonable third party would understand that an agent had authority to act". ("Chatbot says you can submit a form within 90 days to get a retroactive bereavement discount" sounds perfectly reasonable, so the doctrine applies.) https://en.wikipedia.org/wiki/Apparent_authority https://en.wikipedia.org/wiki/Apparent_authority

jasonjayr 3y ago

IANAL, but it's astounding they took that as their defense, rather than pointing to a line (I hope?) in their ToS that says "This agreement is the complete terms of service, and cannot be amended or changed by any agent or representative of the company except by ... (some very specific process the bot can't follow)". I've seen this mentioned in several ToSs, I expect it to be standard boilerplate at this point ...

JCM9 3y ago
Courts often rule that you can’t use ToS to overcome common sense. ToS are not a get out of jail free card if your company just does stupid things.
AlexandrB 3y ago
How is that enforceable? In many cases this is carte blanche for company representatives to lie to you. No one is going to read the ToS and cross reference it with what they're being told in real time. Moreover, if a customer was familiar with the ToS they would not be asking questions like this of a chatbot. The entire idea of having a clause like this while also running a "help" chatbot that can contradict it seems like bad faith dealing.
dataflow 3y ago
How do those clauses actually work? If a rep does something nice for you (like give you something for free), could the airline say it never agreed to that in writing or whatever and demand it back? How are you supposed to know if a rep has authority to enter into an agreement with you over random matters? But, to your question, my guess is that would basically be telling people not to avoid their chatbot, which they don't want to do.
- paxys 3y ago
  It's more to shield them from cases like a rep gifting you free flights for life.
  - dataflow 3y ago

deleted 3y ago

[deleted]

animex 3y ago

Straight out of I, Robot.

skywhopper 3y ago

The claim is so outrageous that I wish there were a way (I assume there probably isn't) for the company or the lawyers to have been sanctioned outside what the plaintiff was asking for.