6 ms·
Tens of thousands of Americas are killed every year in car crashes. The existence of a case or some cases where a self-driving car caused injury has zero value.
by returningfory2 1y ago
Tens of thousands of Americas are killed every year in car crashes. The existence of a case or some cases where a self-driving car caused injury has zero value. What matters is the rate of cases per mile driven.
- timewizard 1y agoThe US drives about 3.2 trillion miles per year. Waymo has 56.7 million miles over several years. Their percentage data is essentially useless. 50% of fatalities involve alcohol or drugs and are often single vehicle accidents. 25% involve youth or inexperience. 15% involve motorcycles. 15% involve pedestrians. What I really need to see is a complete breakdown of every accident a Waymo has had. Then I can start to compare their actual performance to the previously known outcomes.
- p_j_w 1y ago> The US drives about 3.2 trillion miles per year. Waymo has 56.7 million miles over several years. Their percentage data is essentially useless. Your third sentence doesn’t follow from your first two. On what grounds do you draw this conclusion?
- prasadjoglekar 1y agoThis is the statistic I was hoping to see. It wasn't in the paper as best as I looked.
- shermantanktop 1y agoHow would you compare single-incident reports to get to a meaningful conclusion?My guess is that Waymo makes mistakes that a human wouldn't, and vice versa. At that point the overall safety record, which normalizes those differences, seems the most relevant.
- crazygringo 1y ago> The US drives about 3.2 trillion miles per year. Waymo has 56.7 million miles over several years. Their percentage data is essentially useless. No, that's not how statistics works. The percentage data's accuracy depends mainly on the number of incidents recorded (and somewhat on the rate of incidents). But the percentage of the whole is completely irrelevant. If you are basing something on 10 incidents but it's 50% of the total, it's still terrible accuracy. Whereas if you are basing something on 100,000 incidents but it's only 0.1% of the total, it's still going to be quite accurate, assuming the incidents come from the same overall distribution.
- timewizard 1y ago> If you are basing something on 10 incidents but it's 50% of the total, it's still terrible accuracy. The ratio of 3.2 trillion to 56.7 million, which is already incredibly generous to Waymo's position, is 5 orders of magnitude in difference. So any calculations from Waymos data are going to be insanely inaccurate and not something you can extrapolate from. The main, and most obvious case, evidenced by this, is Waymo does not operate where snow falls. Human beings do. We're missing so much of the picture I don't think you can say Waymo's are 75% less accident prone, or 80% less likely to hit a pedestrian. Those are just nonsense numbers.
- achatham 1y agoThe paper under discussion only considers human accidents in similar environments to where Waymo operates. So it's only making a claim about like-for-like driving. You could still say you care about snow driving and want to see that comparison, but it doesn't mean the claims in this paper are wrong.
- DAGdug 1y agoThis! (Thank you for the comment). There’s a reason a 1000 random samples is adequate to reasonably estimate what’s common metrics in a population the size of USA or India (or infinitely large).
- timewizard 1y agoRandom samples of the _same_ user base. If the user base of "waymo riders" and "everyday drivers" does not match then you're not sampling what you think you are.
- DAGdug 1y agoYeah, that’s fair, but is implicit since I’m arguing against the “sample size is inadequate” POV, not the “there are distributional biases in data” POV. There are a gazillion ways to adjust for these biases (ex. propensity score matching) going beyond just user-base but also including weather type, road type, location, time of day, day of week, traffic density, pedestrian density … that can be done easily with far less than the sample size waymo has. And I bet they do these adjustments.
- D-Coder 1y ago> 50% of fatalities involve alcohol or drugs and are often single vehicle accidents. This suggests that Waymo is cutting traffic fatalities by 50% (per million miles) right off the top.
- timewizard 1y agoIf an only if every drunk person decides to take one instead of driving themselves home. Drunk people being known for having exceptionally poor judgement and self awareness. It suggests that they _could_ cut fatalities by that much. Then again, a whole new mode of accident, where the inebriated decide to step out of a moving vehicle and injure themselves that way. This is a dynamic system where human decisions are never fully removed from the loop.
- standardUser 1y ago> Then again, a whole new mode of accident, where the inebriated decide to step out of a moving vehicle and injure themselves that way. If I understand correctly, you believe that the advent of self-driving cars will cause passengers to voluntarily exit a moving vehicle? That sounds like absolute nonsense with no basis in reality.
- Mawr 1y ago> Then again, a whole new mode of accident, where the inebriated decide to step out of a moving vehicle and injure themselves that way. Why would they need to be in a self-driving vehicle to do that?
- SpicyLemonZest 1y agoYou'll be happy to know, then, that the NHTSA publishes the data you're looking for at https://www.nhtsa.gov/laws-regulations/standing-general-order-crash-reporting https://www.nhtsa.gov/laws-regulations/standing-general-orde..., in addition to Waymo's own reporting in their safety hub. For me personally, I find summary data to be more informative than a pile of individual reports, but I hope your comparison goes well!
- bluGill 1y agoI do want to know how Waymo compares to middle ages adults who are not on alcohol/drugs. However that they are better than humans overall is still a big deal even if the data is somewhat suspect by not doing that additional breakdown.
- boulos 1y agoOur safety research team is interested in this topic, too! In a previous study, they've tried to model it: > Building on that, the Collision Avoidance Benchmarking paper presents a novel methodology to evaluate how well autonomous driving systems avoid crashes. The study, which to our knowledge is the first of its kind, introduces a reference model that represents an ideal human state for driving—the response time and evasive action of a human driver that is non-impaired, with eyes always on the conflict (NIEON). Put simply, unlike an average human driver, NIEON is always attentive and doesn’t get distracted or fatigued¹. The data showed that the Waymo Driver outperformed the NIEON human driver model by avoiding more collisions and mitigating serious injury risk in simulated fatal crash scenarios. (From https://waymo.com/blog/2022/09/benchmarking-av-safety https://waymo.com/blog/2022/09/benchmarking-av-safety) AIUI (I'm not on that team), a major challenge is getting good baseline data. Collision reports may not (reliably) capture that kind of data, and it's clearly subjective or often self-reported outside of cases like DUI charges.
- Veserv 1y agoFor which any such system must be assumed unsafe until proven safe and the burden of proof lies with the manufacturer to present sound, robust, transparent, third-party audited evidence. You do not get to counter-argue: “What matters is the rate of cases per mile driven” without actually presenting that number with supporting evidence. Otherwise the only sound conclusion is the default presumption of non-safety. In the case of Waymo, we have some tentative supporting evidence from this and other studies Waymo has run. However, that is still insufficient, even ignoring the lack of audits by non-conflicted parties, to strongly conclude Waymo is safer than a human. The evidence is promising, but it is only prudent to wait for further confirmation. In contrast, Cruise was almost definitely not safer than a human driver. In 2023, Cruise ADS cars drove 2,064,728 miles [1] and were involved in, by my count, 29 collisions with 5 causing injury [2], namely incidents on 2023-05-04, 2023-05-21, 2023-06-09, 2023-08-18, 2023-10-02. That is ~72,000 miles per collision and ~400,000 miles per injury in contrast to the national human averages of ~500,000 per reported collision (which is non-comparable) and ~1,270,000 miles per injury (which is comparable). So, absent a more detailed analysis, Cruise ADS cars were ~3x MORE likely to be involved in a injury causing collision per mile. Details and evidence matter in these discussions. Blanket rhetoric and optimism is not prudent when discussing new safety-critical systems. [1] https://thelastdriverlicenseholder.com/2024/02/03/2023-disengagement-reports-from-california/ https://thelastdriverlicenseholder.com/2024/02/03/2023-disen... [2] https://www.dmv.ca.gov/portal/vehicle-industry-services/autonomous-vehicles/autonomous-vehicle-collision-reports/ https://www.dmv.ca.gov/portal/vehicle-industry-services/auto...
- standardUser 1y agoI'm not sure what you're getting at here. By all accounts, Waymo operates more safely than a human driver. It's operated tens of millions of rides in several major cities over several years, hence all the useful data. There is absolutely nothing stopping the continued expansion of self-driving vehicles, Waymo or otherwise, and there are myriad factors moving it forward rapidly.
- Veserv 1y agoI think I was pretty clear. > You do not get to counter-argue: “What matters is the rate of cases per mile driven” without actually presenting that number with supporting evidence. Otherwise the only sound conclusion is the default presumption of non-safety. I then pointed out how Waymo does present such evidence. But, if you applied that argument to Cruise you would be wrong. That demonstrates how that argument (when not presenting the numbers) can be used to support both good and bad and is thus a bad argument. The correct argument when somebody points to anecdotes of bad outcomes is to present statistically sound data of good outcomes, not argue they did not present statistically sound data of bad outcomes thus you get to assume it is good.
- stefan_ 1y agoThe reason they investigate every airplane crash for a root cause is not necessarily because the number of deaths tilts the statistics, it's because they are highly complicated mechanical and electrical machines where there is always the chance for a systematic fault that will doom many further planes if not found and rectified. The same of course applies to self-driving cars; they are literally cars driven by software, of course you need to do a root cause investigation every time to rule out that it's not a bug in the software that will kill another person (and many after) when the next car happens to go down that rare branch of the system. It's embarrassing to see that the people that call themselves engineers at these companies have not held their work to this standard, and are instead publishing glossy brochures making whacky statistical arguments.
- AlotOfReading 1y agoWhy do you think they don't do root cause investigations? I've personally read through the root cause reports for most of the notable AV accidents. They're not always quite as intensive as aerospace, but I'd be hard pressed to describe any of them as wacky statistical arguments. Obviously most of those reports aren't public, but I'm assuming you also have industry access.
- cratermoon 1y agoKey point: the root cause investigation is not driven by the company involved. The FAA and the NTSB investigate and can by law demand documentation from the company.
- bluGill 1y agoThe real shame is we don't investigate normal human drivers the same way.
- Lammy 1y ago> Tens of thousands of Americas are killed every year in car crashes. CDC “Underlying Cause of Death” dataset sez https://wonder.cdc.gov/ucd-icd10-expanded.html https://wonder.cdc.gov/ucd-icd10-expanded.html https://i.imgur.com/4PB0xyC.jpeg https://i.imgur.com/4PB0xyC.jpeg - “Person injured in unspecified motor-vehicle accident, traffic” is the 50th leading cause of death at 0.4% of deaths. - “Person injured in collision between other specified motor vehicles (traffic)” is the 108th leading cause of death at 0.2% of deaths.
- garrettjoecox 1y agoWas your intent to disprove or back up the claims of parent? 0.4% and 0.2% sound low, but make up for ~110,000 deaths. Spread across a 5 year period does indeed equal “tens of thousands” every year.
- Lammy 1y agoMy intent was to have some real data. Do you only write comments to persuade others? Form your own opinion, because now you have data to inform one.
- 6510 1y agoThe mcdrive is the most dangerous road.
- ahahahahah 1y agoIt sounds like you are refuting the parent's claim. But those two categories add up to ~110,000 deaths over a 6 year period, which seems like can be reasonably described as 10000s per year. But, more importantly, you missed a bunch of relevant categories: V89.2 (Person injured in unspecified motor-vehicle accident, traffic) 80,434 V87.7 (Person injured in collision between other specified motor vehicles (traffic)) 29,982 V09.2 (Pedestrian injured in traffic accident involving other and unspecified motor vehicles) 27,934 V03.1 (Pedestrian injured in collision with car, pick-up truck or van, traffic accident) 15,129 V43.5 (Car occupant injured in collision with car, pick-up truck or van, driver injured in traffic accident) 9,810 V29.9 (Motorcycle rider [any] injured in unspecified traffic accident) 8,410 V29.4 (Driver injured in collision with other and unspecified motor vehicles in traffic accident) 7,688 V47.5 (Car occupant injured in collision with fixed or stationary object, driver injured in traffic accident) 6,379 V49.9 (Car occupant [any] injured in unspecified traffic accident) 6,349 V23.4 (Motorcycle rider injured in collision with car, pick-up truck or van, driver injured in traffic accident) 5,851 V43.6 (Car occupant injured in collision with car, pick-up truck or van, passenger injured in traffic accident) 3,728 V27.4 (Motorcycle rider injured in collision with fixed or stationary object, driver injured in traffic accident) 3,504
- cratermoon 1y agoWhat the discussion below is completely missing, and the point of my link to the incident involving dragging the pedestrian, is the incidence of harm to people not in a vehicle. "The number of pedestrian deaths in the United States is skyrocketing. In 2022 traffic crashes killed 7,805 people on foot—that’s an 83 percent rise from 2009, and a 40-year high. The vast majority of those deaths involved a car colliding into a human"[1] "Pedestrian deaths have been climbing since 2010 because of unsafe infrastructure and the prevalence of SUVs, which tend to be more deadly for pedestrians than smaller cars, according to Martin."[2] The issue with the safety claims of self-driving is focusing on vehicle-to-vehicle interactions and failing to handle the chaotic and unpredictable nature of the environment. There are numerous stories of pathological behavior of self-driving vehicles when encountering simple environmental features that a human driver would handle without a second's hesitation[3]. Pedestrians, cyclists, people using mobility devices, and numerous other non-vehicle road users represent an unaddressed challenge to the safety claims made by Waymo and others. 1 https://slate.com/business/2024/10/cars-suvs-pedestrian-deaths-traffic-safety-lidar-computer-vision.html https://slate.com/business/2024/10/cars-suvs-pedestrian-deat... 2 https://www.npr.org/2023/06/26/1184034017/us-pedestrian-deaths-high-traffic-car https://www.npr.org/2023/06/26/1184034017/us-pedestrian-deat... 3 https://www.npr.org/2023/08/26/1195695051/driverless-cars-san-francisco-waymo-cruise https://www.npr.org/2023/08/26/1195695051/driverless-cars-sa...