8 ms·

Junk science is a huge problem in law enforcement. Treating lie detectors as canonical, the blood splatter analysis to recreate crime scenes, this kind of stuf

Junk science is a huge problem in law enforcement. Treating lie detectors as canonical, the blood splatter analysis to recreate crime scenes, this kind of stuff. It is especially problematic because police orgs are generally pretty good at adopting new techniques. But unfortunately the marketplace of ideas they have to shop in suffers from overrepresentation of garbage products. A huge part of law enforcement enhancement is going to involve cleaning this market up.

loteck 4y ago
Let's make sure we add "gunshot detection" to this list, since that is the currently popular for-profit junk science product that police are pouring millions of dollars into acquiring all over the US.
- wrigby 4y ago
  I’d be interested in understanding this better - what kind of sensors do you have in mind that are junk?
  - dsfyu404ed 4y ago
    They use microphones to detect gunshots. It works in a controlled environment but in an actual municipality the data needs enough "massaging" by human analysts to get what are still fundamentally low-quality tips it's somewhere between redundant and a waste of resources.
    - cratermoon 4y ago
      > the data needs enough "massaging" by human analysts to get what are still fundamentally low-quality tips it's somewhere between redundant and a waste of resources. And those analysts are part of a law enforcement community that is rife with biases about crime and guns, which means most of the places where the system finds gunshots are where the biased people (who by default placed the detectors according to a bias) expect to hear gunshots.
  - BryantD 4y ago
    ShotSpotter’s results are iffy, which appears to be a combination of a) ShotSpotter employees overriding the AI and b) weaknesses in the tech. There’s also some potential for reinforcing bias if you only put the sensors in neighborhoods you’re already worried about. https://news.ycombinator.com/item?id=28264686 https://news.ycombinator.com/item?id=28264686 has previous discussion on this from a number of viewpoints.
    - googlryas 4y ago
      The goal of police is public safety, and nothing to do with preventing reinforcing biases. If you designed a polar bear detector to help prevent people from being eaten by polar bears, would you also deploy them in Florida for the sake of fairness, even though there are no polar bear attacks there?
      - RHSeeger 4y ago
        > The goal of police is public safety To be clear... the goal of the police _force_ is to enforce the law. The goal of (some) individual police officers is public safety, but certainly not all of them (and some days, it seems like not many of them... but you generally only hear about the bad stuff in the news, so take that with a grain of salt).
        Quinner 4y ago
        The goal of the police is to have secure jobs with a decent salary and benefits, and ideally be meaningful in some way (could be promoting public safety... could be exerting power over others). Often enforcing the law helps secure budgets (especially when it comes to enforcing drug laws with lucrative civil forfeiture seizures), but more recently many police departments have tried a different tactic when budgets have been threatened: actively refusing to enforce the law, in order to increase crime and prompt political change.
        dinkumthinkum 4y ago
        The police increase crime?
  - weakfortress 4y ago
    If you've ever shot a gun or been around guns you'd know the rapport of various guns is not always consistent. Often, it is difficult to tell the difference between a gunshot and a firework for example. Even something like dropping a wood pallet far enough away can sound like the echos of a gunshot. Backfiring from a car can sound remarkably close to a small caliber rifle. If a gunshot doesnt happen quite literally within a few dozen feet of you it's often hard to tell. Typically the way you can tell is people who shoot guns generally don't do it once. Several rapports tend to raise the probability of a gun. ShotSpotter works in isolation because in isolation a certain noise profile is almost certainly a gun shot. Like most ML aiming to change the world it's powered by p-hacking.
    - selimthegrim 4y ago
      the report, rapport is not something I want to have with a gun unless I am a character in Full Metal Jacket
- mhb 4y ago
  Flock Safety Raven Gunshot Detection and Location System Combine gunshot detection with ALPR evidence to drive better outcomes https://www.flocksafety.com/products/raven https://www.flocksafety.com/products/raven
Sakos 4y ago
> Junk science is a huge problem in law enforcement. Treating lie detectors as canonical, the blood splatter analysis to recreate crime scenes, this kind of stuff. I'm not sure what's the bigger problem. The use of junk science by law enforcement or how law enforcement and their methods are portrayed by popular media that ensure the use of junk science (and other harmful methods) can persist long-term. I'm sure everybody and their mother has seen Dexter (who works as a blood splatter analyst) or any number of shows/movies or all the news where police dogs are portrayed in an incredibly positive light (I'm looking at you, Paw Patrol).
- watwut 4y ago
  Those are at least fiction. I have seen documentaries where pseudo psychology (not holding eye contact means you are lying, this or that entirely normal behavior treated as something chillingly sinister, parent deemed nit sufficiently crying) was treated as sure clue of guilt. And gut feelings, gut feelings of detectives were literally treated as evidence in those documentaries
  - scarmig 4y ago
    Most documentaries should be considered a subgenre of fiction. Inspired by reality, but mostly interested in telling a compelling narrative.
  - britneybitch 4y ago
    They're not always treated as fiction though. Some police officers model their behavior after what they see on TV. And police departments have given shows access to locations and props in exchange for favorable portrayal. John Oliver did a show about it recently. https://www.youtube.com/watch?v=DNy6F7ZwX8I https://www.youtube.com/watch?v=DNy6F7ZwX8I
    - cratermoon 4y ago
      LA PD essentially invented the cop show. https://timeline.com/dragnet-lapd-propaganda-cop-bb19d9a5fb6f https://timeline.com/dragnet-lapd-propaganda-cop-bb19d9a5fb6...
nucleogenesis 4y ago
Yeah this isn’t surprising considering these are the people who think touching or breathing near fentanyl can kill you
feet 4y ago
It doesn't help that law enforcement organizations seem to higher the dullest knives in the drawer, of course they can't tell bs from non-bs. Whatever confirms their biases gets green-lit
- anigbrowl 4y ago
  The thing is that prosecutors and courts are supposedly harboring some of the sharpest knives in the drawer, yet they cheerfully accept and often pass on BS at face value when it's expedient to do so.
  - feet 4y ago
    Supposedly is doing a lot of heavy lifting here, considering the state of the system I have a hard time believing it
  - sobkas 4y ago
    > The thing is that prosecutors and courts are supposedly harboring some of the sharpest knives in the drawer, yet they cheerfully accept and often pass on BS at face value when it's expedient to do so. Court system have one basic goal, to be a test if accused party have enough money to spend on lawyers, experts and their expertise. Affluenza much?
schwartzworld 4y ago
Citations Needed did an episode a few years back on how TV Copaganda represents forensics as a much more solid "science" than it actually is. https://citationsneeded.libsyn.com/episode-94-the-goofy-pseudoscience-copaganda-of-tv-forensics https://citationsneeded.libsyn.com/episode-94-the-goofy-pseu...
Waterluvian 4y ago
I don’t think it’s that the market is full of garbage. There is a demand from law enforcement for the garbage. They want science tools to confirm their biases and broken practices.
- 8note 4y ago
  Not really science tools. They want authoritative looking tools to confirm their biases and broken practices. A scientific tool could prove a different direction than what law enforcement wants it to say.
Spooky23 4y ago
Agree. The problem imo is that cops have an insular culture and are trained to control situations. That control is often an Illusion. They are also often exempt from some of the procurement rules that apply to other government entities. So they tend to accept handshake claims from people identified as “good guys” through some connection or the salesman’s former on the job status. Another issue is that they don’t pay for or maintain technology in many cases, and the organizations tend to want to fund overtime, gas and bullets over all else. Police rely on grants to buy stuff with tight timelines. So there’s a tendency to blow the federal grant quickly on something cool that some “good guy” promises.
thecrash 4y ago
"It is difficult to get a man to understand something when his salary depends on his not understanding it." - Upton Sinclair
akudha 4y ago
I am terrified of affecting someone's health, freedom or money (especially health and freedom) unknowingly, mistakenly etc. How do people who lie in court, use unproven tools etc to screw someone's entire life sleep at night? This is so far beyond my comprehension.
- deleted 4y ago
  [deleted]
- 20after4 4y ago
  They convince themself that the person is really guilty (of something, maybe not even the thing they are accused of, just generally bad / evil / wrong color / wrong religion. really whatever it takes for them to believe someone deserves punishment) .... then the ends justify the means.
- Bluestrike2 4y ago
  Well, for starters, they don't actually think their junk science is junk science. If you're confident that you were right all along, and you nailed the "bad guy," you're going to sleep just fine. All of these people believe they're helping further the cause of justice; very few go to work thinking "this is bullshit, I know it, and I don't care." Given their training and experience--even though it's experience in a field we later define as junk science--they become confident in their own work, and confirmation bias is omnipresent. Not just for individual cases, but for entire forensic fields and methods. It's not exactly the same, but there have been countless examples of prosecutors and police officers who, when faced with convictions overturned due to DNA evidence, continued to maintain that the person was nevertheless guilty despite that evidence. And if I'm being totally honest, I can't even imagine how tempting that fantasy would be. The alternative is acknowledging that you got it wrong and an innocent person paid an extraordinary price for your failure, losing years of their life behind bars. Worse yet, your failure allowed allowed the actual murderer/rapist/etc. to get off scot-free and potentially harm others. It's terrifying enough to imagine screwing up a single case. Now imagine screwing up hundreds or possibly thousands, after spending years using what you believed to be forensic science that turned out to be little more than bullshit. In that light, the temptation to search for alternative explanations, fight against information and evidence that contradicts your theory of the case, etc. has to be overwhelming. And while there are absolutely people in the justice system who don't succumb to that temptation, there are plenty who do.
partdavid 4y ago
The list goes on: instant-drug tests used for traffic stops with undeclared and uninvestigated false positive rates, drug sniffing dogs (cf. https://reason.com/2021/05/13/the-police-dog-who-cried-drugs-at-every-traffic-stop/ https://reason.com/2021/05/13/the-police-dog-who-cried-drugs... for example), burn patterns in arson investigation, etc.

loteck 4y ago

Let's make sure we add "gunshot detection" to this list, since that is the currently popular for-profit junk science product that police are pouring millions of dollars into acquiring all over the US.

wrigby 4y ago
I’d be interested in understanding this better - what kind of sensors do you have in mind that are junk?
- dsfyu404ed 4y ago
  They use microphones to detect gunshots. It works in a controlled environment but in an actual municipality the data needs enough "massaging" by human analysts to get what are still fundamentally low-quality tips it's somewhere between redundant and a waste of resources.
  - cratermoon 4y ago
    > the data needs enough "massaging" by human analysts to get what are still fundamentally low-quality tips it's somewhere between redundant and a waste of resources. And those analysts are part of a law enforcement community that is rife with biases about crime and guns, which means most of the places where the system finds gunshots are where the biased people (who by default placed the detectors according to a bias) expect to hear gunshots.
- BryantD 4y ago
  ShotSpotter’s results are iffy, which appears to be a combination of a) ShotSpotter employees overriding the AI and b) weaknesses in the tech. There’s also some potential for reinforcing bias if you only put the sensors in neighborhoods you’re already worried about. https://news.ycombinator.com/item?id=28264686 https://news.ycombinator.com/item?id=28264686 has previous discussion on this from a number of viewpoints.
  - googlryas 4y ago
    The goal of police is public safety, and nothing to do with preventing reinforcing biases. If you designed a polar bear detector to help prevent people from being eaten by polar bears, would you also deploy them in Florida for the sake of fairness, even though there are no polar bear attacks there?
    - RHSeeger 4y ago
      > The goal of police is public safety To be clear... the goal of the police _force_ is to enforce the law. The goal of (some) individual police officers is public safety, but certainly not all of them (and some days, it seems like not many of them... but you generally only hear about the bad stuff in the news, so take that with a grain of salt).
      - Quinner 4y ago
        The goal of the police is to have secure jobs with a decent salary and benefits, and ideally be meaningful in some way (could be promoting public safety... could be exerting power over others). Often enforcing the law helps secure budgets (especially when it comes to enforcing drug laws with lucrative civil forfeiture seizures), but more recently many police departments have tried a different tactic when budgets have been threatened: actively refusing to enforce the law, in order to increase crime and prompt political change.
        dinkumthinkum 4y ago
        The police increase crime?
- weakfortress 4y ago
  If you've ever shot a gun or been around guns you'd know the rapport of various guns is not always consistent. Often, it is difficult to tell the difference between a gunshot and a firework for example. Even something like dropping a wood pallet far enough away can sound like the echos of a gunshot. Backfiring from a car can sound remarkably close to a small caliber rifle. If a gunshot doesnt happen quite literally within a few dozen feet of you it's often hard to tell. Typically the way you can tell is people who shoot guns generally don't do it once. Several rapports tend to raise the probability of a gun. ShotSpotter works in isolation because in isolation a certain noise profile is almost certainly a gun shot. Like most ML aiming to change the world it's powered by p-hacking.
  - selimthegrim 4y ago
    the report, rapport is not something I want to have with a gun unless I am a character in Full Metal Jacket
mhb 4y ago
Flock Safety Raven Gunshot Detection and Location System Combine gunshot detection with ALPR evidence to drive better outcomes https://www.flocksafety.com/products/raven https://www.flocksafety.com/products/raven

Sakos 4y ago

> Junk science is a huge problem in law enforcement. Treating lie detectors as canonical, the blood splatter analysis to recreate crime scenes, this kind of stuff. I'm not sure what's the bigger problem. The use of junk science by law enforcement or how law enforcement and their methods are portrayed by popular media that ensure the use of junk science (and other harmful methods) can persist long-term. I'm sure everybody and their mother has seen Dexter (who works as a blood splatter analyst) or any number of shows/movies or all the news where police dogs are portrayed in an incredibly positive light (I'm looking at you, Paw Patrol).

watwut 4y ago
Those are at least fiction. I have seen documentaries where pseudo psychology (not holding eye contact means you are lying, this or that entirely normal behavior treated as something chillingly sinister, parent deemed nit sufficiently crying) was treated as sure clue of guilt. And gut feelings, gut feelings of detectives were literally treated as evidence in those documentaries
- scarmig 4y ago
  Most documentaries should be considered a subgenre of fiction. Inspired by reality, but mostly interested in telling a compelling narrative.
- britneybitch 4y ago
  They're not always treated as fiction though. Some police officers model their behavior after what they see on TV. And police departments have given shows access to locations and props in exchange for favorable portrayal. John Oliver did a show about it recently. https://www.youtube.com/watch?v=DNy6F7ZwX8I https://www.youtube.com/watch?v=DNy6F7ZwX8I
  - cratermoon 4y ago
    LA PD essentially invented the cop show. https://timeline.com/dragnet-lapd-propaganda-cop-bb19d9a5fb6f https://timeline.com/dragnet-lapd-propaganda-cop-bb19d9a5fb6...

nucleogenesis 4y ago

Yeah this isn’t surprising considering these are the people who think touching or breathing near fentanyl can kill you

feet 4y ago

It doesn't help that law enforcement organizations seem to higher the dullest knives in the drawer, of course they can't tell bs from non-bs. Whatever confirms their biases gets green-lit

anigbrowl 4y ago
The thing is that prosecutors and courts are supposedly harboring some of the sharpest knives in the drawer, yet they cheerfully accept and often pass on BS at face value when it's expedient to do so.
- feet 4y ago
  Supposedly is doing a lot of heavy lifting here, considering the state of the system I have a hard time believing it
- sobkas 4y ago
  > The thing is that prosecutors and courts are supposedly harboring some of the sharpest knives in the drawer, yet they cheerfully accept and often pass on BS at face value when it's expedient to do so. Court system have one basic goal, to be a test if accused party have enough money to spend on lawyers, experts and their expertise. Affluenza much?

schwartzworld 4y ago

Citations Needed did an episode a few years back on how TV Copaganda represents forensics as a much more solid "science" than it actually is. https://citationsneeded.libsyn.com/episode-94-the-goofy-pseudoscience-copaganda-of-tv-forensics https://citationsneeded.libsyn.com/episode-94-the-goofy-pseu...

Waterluvian 4y ago

I don’t think it’s that the market is full of garbage. There is a demand from law enforcement for the garbage. They want science tools to confirm their biases and broken practices.

8note 4y ago
Not really science tools. They want authoritative looking tools to confirm their biases and broken practices. A scientific tool could prove a different direction than what law enforcement wants it to say.

Spooky23 4y ago

Agree. The problem imo is that cops have an insular culture and are trained to control situations. That control is often an Illusion. They are also often exempt from some of the procurement rules that apply to other government entities. So they tend to accept handshake claims from people identified as “good guys” through some connection or the salesman’s former on the job status. Another issue is that they don’t pay for or maintain technology in many cases, and the organizations tend to want to fund overtime, gas and bullets over all else. Police rely on grants to buy stuff with tight timelines. So there’s a tendency to blow the federal grant quickly on something cool that some “good guy” promises.

thecrash 4y ago

"It is difficult to get a man to understand something when his salary depends on his not understanding it." - Upton Sinclair

akudha 4y ago

I am terrified of affecting someone's health, freedom or money (especially health and freedom) unknowingly, mistakenly etc. How do people who lie in court, use unproven tools etc to screw someone's entire life sleep at night? This is so far beyond my comprehension.

deleted 4y ago
[deleted]
20after4 4y ago
They convince themself that the person is really guilty (of something, maybe not even the thing they are accused of, just generally bad / evil / wrong color / wrong religion. really whatever it takes for them to believe someone deserves punishment) .... then the ends justify the means.
Bluestrike2 4y ago
Well, for starters, they don't actually think their junk science is junk science. If you're confident that you were right all along, and you nailed the "bad guy," you're going to sleep just fine. All of these people believe they're helping further the cause of justice; very few go to work thinking "this is bullshit, I know it, and I don't care." Given their training and experience--even though it's experience in a field we later define as junk science--they become confident in their own work, and confirmation bias is omnipresent. Not just for individual cases, but for entire forensic fields and methods. It's not exactly the same, but there have been countless examples of prosecutors and police officers who, when faced with convictions overturned due to DNA evidence, continued to maintain that the person was nevertheless guilty despite that evidence. And if I'm being totally honest, I can't even imagine how tempting that fantasy would be. The alternative is acknowledging that you got it wrong and an innocent person paid an extraordinary price for your failure, losing years of their life behind bars. Worse yet, your failure allowed allowed the actual murderer/rapist/etc. to get off scot-free and potentially harm others. It's terrifying enough to imagine screwing up a single case. Now imagine screwing up hundreds or possibly thousands, after spending years using what you believed to be forensic science that turned out to be little more than bullshit. In that light, the temptation to search for alternative explanations, fight against information and evidence that contradicts your theory of the case, etc. has to be overwhelming. And while there are absolutely people in the justice system who don't succumb to that temptation, there are plenty who do.

partdavid 4y ago

The list goes on: instant-drug tests used for traffic stops with undeclared and uninvestigated false positive rates, drug sniffing dogs (cf. https://reason.com/2021/05/13/the-police-dog-who-cried-drugs-at-every-traffic-stop/ https://reason.com/2021/05/13/the-police-dog-who-cried-drugs... for example), burn patterns in arson investigation, etc.