8 ms·

Waymo reports teen riders for bad behavior and delivers them to the police

60 points by pilingual 2mo ago

thatmf 2mo ago
As Benn Jordan pointed out, imagine what a hacker group or ICE could do with this. Fun times!
worik 2mo ago
In my country (New Zealand) this would be very illegal You cannot detain someone because you do not like their behavior, in almost all situations. In civilised countries It is called kidnapping
tejusarora 2mo ago
My main concern is the deceit, "[s]topping for mechanical issues". Nah bruh
Fedot 2mo ago
AI is just a tool. The real responsibility belongs to the developers, companies, and users who decide how it's built and used—not the AI itself. Anyway, that's insane. But if they were trying to damage the car, I can understand why Waymo reported them to the police.
sameers 2mo ago
https://archive.ph/yUFqG https://archive.ph/yUFqG The car sent the San Mateo PD a report that the riders had toy projectile emitters (aka "guns," I suppose) they were using to shoot pellets at passers-by. It pulled into a parking lot until cops arrived. It doesn't report if it locked the riders in. I was just thinking I want to try at least one joyride but no, now I'm not even interested in giving them those $20.
- rolph 2mo ago
  hmmm kidknapped due to a false positive would be a bit of an oops to try and rewind. its bad enough that the cops do it [just try and get them to give you a ride back to where thay took you from, even when they are told to cut you loose] it causes real damages when you basically abduct and strand someone. these "kids" were shooting orbees out the windows, so mischief on them, but get it wrong once and it could get expensive.
  - wmf 2mo ago
    You're not stranded... just call a Waymo.
  - JumpCrisscross 2mo ago
    > these "kids" were shooting orbees out the windows, so mischief on them, but get it wrong once and it could get expensive It would be a tiny settlement, particularly now that we have documented cases like this one where the policy is preventing real harm. Like, if you're a kid drinking and shooting bystanders from the back of an Uber, what do you think is going to happen?
    - clipsy 2mo ago
      > Like, if you're a kid drinking and shooting bystanders from the back of an Uber, what do you think is going to happen? I certainly don't think the Uber driver would hold me prisoner.
      - rolph 2mo ago
        im thinking a human driver would WTF and pull over into a parking the instant the BB guns came out
        JumpCrisscross 2mo ago
        And provide footage to Uber and the authorities. I don’t want to be in an Uber with a driver who wouldn’t.
- deleted 2mo ago
  [deleted]
- dlcarrier 2mo ago
  Even if there was a way to remotely engage child lock, the front doors are still always unlocked from the inside.
  - fragmede 2mo ago
    The newer vehicles with the suicide doors don't have an emergency exit lever that I could find.
- aaron695 2mo ago
  [dead]
- kylecazar 2mo ago
  I suspect they weren't locked in and stayed because they didn't know the police were coming to their location. Locking people in a car, remotely, as a private company, for "bad behavior", would be a mess.
Schiendelman 2mo ago
I'm 99% sure that a human made these decisions after viewing the cameras in the vehicle, not that the Waymo vehicle itself decided to contact the police and park in a parking lot with these kids. This honestly seems pretty ideal. In a dangerous situation, the only risk is property damage - there's no driver to threaten!
- 4d4m 2mo ago
  Your view is dangerous because you don't see how this can be used to harm you.
- JohnFen 2mo ago
  > This honestly seems pretty ideal. I disagree. A decent response would have been for the car to simply refuse to continue the ride rather than kidnapping the ne'er-do-wells. That Waymo did this means that the hesitations I had about using the service are amplified by a few orders of magnitude. I'm not willing to risk someone at Waymo (or, worse, a machine at Waymo) making the wrong call and kidnapping me by mistake.
  - sidibe 2mo ago
    > A decent response would have been for the car to simply refuse to continue the ride rather than kidnapping the ne'er-do-wells. It seems that's what exactly what they did? Except also called the police which seems the responsible thing to do when dropping drunk kids off somewhere
- deleted 2mo ago
  [deleted]
- baranul 2mo ago
  Agree. AI is not there yet, but it seems it is coming, and that would be scary. People will one day be in the scenario, where AI judges human behavior, and then decides what type of punishments to dish out based on it.
  - nisegami 2mo ago
ButlerianJihad 2mo ago
One time I was settling into a long Waymo ride (about 15 miles through traffic) and it was rolling down a wide urban street where a white English-speaking guy like me is distinctly a minority. And I was playing Pat Benatar's "Hit Me with Your Best Shot" with the windows wide open, and waving at other cars, and singing along at the top of my lungs, and eventually, Waymo Support rings in, and the guy goes, "just checking in! Is everything alright there?" and I go "er... yeah" and he goes "OK! thank you bye!" So I decided maybe there had been an external complaint, and I should perhaps keep the windows up, and the music down a few notches from now on. Especially as a minority in such a neighborhood like that one.
- Schiendelman 2mo ago
  Ha! I don't think there was an external complaint. I suspect that was the mic/cameras IN the car alerting support because you were loud and they wanted to make sure you were okay. :)
  - nojvek 2mo ago
    If you’re screaming for help vs singing on top of your lungs, it’s a minor difference. Worth a check.
    - ButlerianJihad 2mo ago
      Actually, Waymo explicitly notes that their mics are never activated until Support is on the line (and this is a very noticeable event) even though the video cameras are always monitoring. I mean, the way I was singing, they could probably guess at the audio! So there is no way that Support heard or listened to the music or singing in-car. In fact, their own material says "mics are off... so sing your heart out!" So I feel like it was some pedestrian who was perhaps racist or just offended that a Waymo passenger would be enjoying himself so much with Pat Benatar blasting, and just called-in a complaint. It seems the most prosaic explanation. I couldn't explain why Support would proactively just call in "for no reason" without anyone actually complaining about my particular vehicle.

60 points by pilingual 2mo ago

As Benn Jordan pointed out, imagine what a hacker group or ICE could do with this. Fun times!

worik 2mo ago

In my country (New Zealand) this would be very illegal You cannot detain someone because you do not like their behavior, in almost all situations. In civilised countries It is called kidnapping

tejusarora 2mo ago

My main concern is the deceit, "[s]topping for mechanical issues". Nah bruh

Fedot 2mo ago

AI is just a tool. The real responsibility belongs to the developers, companies, and users who decide how it's built and used—not the AI itself. Anyway, that's insane. But if they were trying to damage the car, I can understand why Waymo reported them to the police.

sameers 2mo ago

https://archive.ph/yUFqG https://archive.ph/yUFqG The car sent the San Mateo PD a report that the riders had toy projectile emitters (aka "guns," I suppose) they were using to shoot pellets at passers-by. It pulled into a parking lot until cops arrived. It doesn't report if it locked the riders in. I was just thinking I want to try at least one joyride but no, now I'm not even interested in giving them those $20.

rolph 2mo ago
hmmm kidknapped due to a false positive would be a bit of an oops to try and rewind. its bad enough that the cops do it [just try and get them to give you a ride back to where thay took you from, even when they are told to cut you loose] it causes real damages when you basically abduct and strand someone. these "kids" were shooting orbees out the windows, so mischief on them, but get it wrong once and it could get expensive.
- wmf 2mo ago
  You're not stranded... just call a Waymo.
- JumpCrisscross 2mo ago
  > these "kids" were shooting orbees out the windows, so mischief on them, but get it wrong once and it could get expensive It would be a tiny settlement, particularly now that we have documented cases like this one where the policy is preventing real harm. Like, if you're a kid drinking and shooting bystanders from the back of an Uber, what do you think is going to happen?
  - clipsy 2mo ago
    > Like, if you're a kid drinking and shooting bystanders from the back of an Uber, what do you think is going to happen? I certainly don't think the Uber driver would hold me prisoner.
    - rolph 2mo ago
      im thinking a human driver would WTF and pull over into a parking the instant the BB guns came out
      - JumpCrisscross 2mo ago
        And provide footage to Uber and the authorities. I don’t want to be in an Uber with a driver who wouldn’t.
deleted 2mo ago
[deleted]
dlcarrier 2mo ago
Even if there was a way to remotely engage child lock, the front doors are still always unlocked from the inside.
- fragmede 2mo ago
  The newer vehicles with the suicide doors don't have an emergency exit lever that I could find.
aaron695 2mo ago
[dead]
kylecazar 2mo ago
I suspect they weren't locked in and stayed because they didn't know the police were coming to their location. Locking people in a car, remotely, as a private company, for "bad behavior", would be a mess.

Schiendelman 2mo ago

I'm 99% sure that a human made these decisions after viewing the cameras in the vehicle, not that the Waymo vehicle itself decided to contact the police and park in a parking lot with these kids. This honestly seems pretty ideal. In a dangerous situation, the only risk is property damage - there's no driver to threaten!

4d4m 2mo ago
Your view is dangerous because you don't see how this can be used to harm you.
JohnFen 2mo ago
> This honestly seems pretty ideal. I disagree. A decent response would have been for the car to simply refuse to continue the ride rather than kidnapping the ne'er-do-wells. That Waymo did this means that the hesitations I had about using the service are amplified by a few orders of magnitude. I'm not willing to risk someone at Waymo (or, worse, a machine at Waymo) making the wrong call and kidnapping me by mistake.
- sidibe 2mo ago
  > A decent response would have been for the car to simply refuse to continue the ride rather than kidnapping the ne'er-do-wells. It seems that's what exactly what they did? Except also called the police which seems the responsible thing to do when dropping drunk kids off somewhere
deleted 2mo ago
[deleted]
baranul 2mo ago
Agree. AI is not there yet, but it seems it is coming, and that would be scary. People will one day be in the scenario, where AI judges human behavior, and then decides what type of punishments to dish out based on it.
- nisegami 2mo ago

ButlerianJihad 2mo ago

One time I was settling into a long Waymo ride (about 15 miles through traffic) and it was rolling down a wide urban street where a white English-speaking guy like me is distinctly a minority. And I was playing Pat Benatar's "Hit Me with Your Best Shot" with the windows wide open, and waving at other cars, and singing along at the top of my lungs, and eventually, Waymo Support rings in, and the guy goes, "just checking in! Is everything alright there?" and I go "er... yeah" and he goes "OK! thank you bye!" So I decided maybe there had been an external complaint, and I should perhaps keep the windows up, and the music down a few notches from now on. Especially as a minority in such a neighborhood like that one.

Schiendelman 2mo ago
Ha! I don't think there was an external complaint. I suspect that was the mic/cameras IN the car alerting support because you were loud and they wanted to make sure you were okay. :)
- nojvek 2mo ago
  If you’re screaming for help vs singing on top of your lungs, it’s a minor difference. Worth a check.
  - ButlerianJihad 2mo ago
    Actually, Waymo explicitly notes that their mics are never activated until Support is on the line (and this is a very noticeable event) even though the video cameras are always monitoring. I mean, the way I was singing, they could probably guess at the audio! So there is no way that Support heard or listened to the music or singing in-car. In fact, their own material says "mics are off... so sing your heart out!" So I feel like it was some pedestrian who was perhaps racist or just offended that a Waymo passenger would be enjoying himself so much with Pat Benatar blasting, and just called-in a complaint. It seems the most prosaic explanation. I couldn't explain why Support would proactively just call in "for no reason" without anyone actually complaining about my particular vehicle.