5 ms·

How Well Do We Understand Probabilities in Medicine? (2016)

53 points by bladecatcher 8y ago

progval 8y ago
Sadly unsurprising. I remember reading a conversation on Twitter where a psychologist wrote openly "I don't use statistics because I have an analytical orientation".
tcj_phx 8y ago
I find it ironic that this article is posted on a website aimed at the mental health industry. The actual challenge faced by mental health practitioners is with the futility of their approach. I met a young man (early 20's) who'd recently arrived in town for the purposes of "recovery". Having learned something about people's problems with self-medicating I asked, "People usually know when their problems with substances started. When did it start for you?" He instantly said that when he was in third grade his teacher thought he was disruptive. Parents took him to the doctor, who prescribed medication. When he got to 9th grade he gave his parents an ultimatum: "if you don't take me off these drugs, I'm going to kill myself." His parents promptly took him off the drugs that didn't address the boredom he'd experienced in his suffocating 3rd grade classroom, and that's when he started self-medicating with whatever he could get his hands on. The DSM is the mental health industry's guide to help practitioners precisely diagnose symptoms. The problem is the industry frequently jumps to treatment without concern as to the cause of the patients' symptoms. Court-ordered treatment (aka "assisted outpatient treatment") is where the mental health industry decides to force a patient to take the drugs they think they need. I have the affidavits that were filed against my friend. They say, essentially, "Patient expects us to believe that her symptoms are related to consuming 2 bottles of liquor a day. She is clearly in need of court-ordered treatment because she does not believe that she has a chronic condition. We know that she has a chronic condition because she's had two previous orders for treatment. She took herself off our forced services 4 years ago, and here she is again." Before this went down, I'd decided that my friend's substance abuse problems were related to her having been adopted, exposed to meth amphetamine as a pre-teen, having "lost her future" at 16 years old when she was expelled for self-medicating with what is now approved as a breakthrough therapy for PTSD (MDMA), and having been injected with a prescription endocrine disruptor [0] with the black-box warning of "cortisol deficiency" at 18 or 19 years old. This "birth control" drug is known to make some women suicidally-depressed. Cortisol deficiency is now associated with psychosis. My friend allowed that maybe the injections took her from "drug abuser" to "drug addict". [0] https://en.wikipedia.org/wiki/Depo-Provera https://en.wikipedia.org/wiki/Depo-Provera In the world where mental health professionals address causes rather than symptoms and treat conservatively when the causes of a condition are unknown, those doctors would have recognized my friend's presentation as a form of substance-induced psychosis [1], and provided support to help her sober up. [1] https://en.wikipedia.org/wiki/Substance-induced_psychosis https://en.wikipedia.org/wiki/Substance-induced_psychosis Rather than provide rational treatment, they just force her to take "anti-psychotics" in the delusional belief that their drugs (rather than sobriety) are what allows my friend to be functional. At one point they thought she needed two different drugs. My friend explained her presentation as, "I'm sorry, these drugs make us slow". Every medical specialty has practices that aren't actually justified by the findings of science. I believe Psychiatry is the most important of the medical specialties, but the path "mainstream" practitioners has taken since the 1950's has been a mistake: most psychiatric drugs are just modern FDA-approved "patent medicines" [2] that don't actually address the causes of the patient's symptoms. While some people like their psych drugs, they'd probably do much better with more scientific approach to their complaints. [2] https://en.wikipedia.org/wiki/Patent_medicine https://en.wikipedia.org/wiki/Patent_medicine
- CWSZ 8y ago
  Rather than provide rational treatment, they just force her to take "anti-psychotics" in the delusional belief that their drugs (rather than sobriety) are what allows my friend to be functional. My best friend's father is a paranoid schizophrenic. Antipsychotic medication allows him to live a relatively normal life.
  - tcj_phx 8y ago
    > My best friend's father is a paranoid schizophrenic. Antipsychotic medication allows him to live a relatively normal life. How do you know that it's the medication that allows him to "live a relatively normal life", and not the other factors? If you want to trade anecdotes... My other friend's father fell out of a tree and broke his back when she was a child (iirc). Some time later they decided he was a "schizophrenic". He's been on antipsychotics ever since. His condition has spiraled downward over the subsequent years. Last I heard he was full-on crazy. If the doctors had provided supportive treatment instead of suppressive, his life trajectory would have been totally different. My aunt's good friend had a "psychotic break" soon after her husband died unexpectedly. The friend was put on "anti-psychotics". She's now dying of liver failure, certainly as a consequence of her long-term drugging. The ugly truth about so-called "antipsychotic medication" is that the class is palliative rather than curative, and actually causes the deterioration it supposedly treats. Robert Whitaker makes the case that medications turns people's "episodes" into chronic conditions: https://www.madinamerica.com/2016/07/the-case-against-antipsychotics/ https://www.madinamerica.com/2016/07/the-case-against-antips... and https://www.madinamerica.com/2018/03/a-tale-of-two-studies/ https://www.madinamerica.com/2018/03/a-tale-of-two-studies/ and https://www.madinamerica.com/2017/09/thou-shall-not-criticize-our-drugs/ https://www.madinamerica.com/2017/09/thou-shall-not-criticiz... , for example.
    - CWSZ 8y ago
      I dislike exchanging anecdotes too. If you can point me to a treatment for schizophrenia tested in quality randomized trials with the success rate of antipsychotics (which still have a very poor success rate), please do.
mrnobody_67 8y ago
I always wonder the pros/cons of getting tested for no specific symptom... anxiety.stress.costs of false positives could be enormous. There's even a book & several articles out there arguing that getting tested for cancer might be a bad idea: https://www.theatlantic.com/health/archive/2015/06/should-I-get-tested-screened-for-cancer/397100/ https://www.theatlantic.com/health/archive/2015/06/should-I-...
iheartpotatoes 8y ago
BSEE and MSEE with 8 years of heavy math and the fact that I still have to read the solution slowly and not really get it scares the bejeezers out of me.
- hugey010 8y ago
  If you have any kind of test that seems relatively accurate (5% failure rate), 1 in every 20 tests will be identified as a positive candidate no matter the actual incidence rate. When applied over a huge population the number of positive candidates is going to also be huge. Since the incidence rate in this scenario is so low (1/1000) the odds of a true positive selected from the positive candidates is also very low.
- tnecniv 8y ago
  This is a little surprising to me. I was required to take a probability class for my BS in electrical engineering, and a version of this problem was a homework question.
  - iheartpotatoes 8y ago
    Funny you should call this out: As an undergrad I got to choose between a prob & statistics path and a complex math / advance calc path. I took the complex math path. However, I was required to take a semester on engineering methods that touched on statistics, but that was part of one semester. As a grad student, I went down the numerical methods path for parallel computing. Not as many classes as undergrad, but still. College: RPI. Graduation year: '91.
DanBC 8y ago
Bit annoying that this article hat-tips Taleb, when Gerd Gigerenzer has been doing this better and for longer. This book is a good introduction, includes a lot of anecdote from different health care professionals across a range of different testing (including cancer and HIV), and mentions the research across different health care professionals. https://www.amazon.com/Reckoning-Risk-Learning-Live-Uncertainty/dp/0140297863/ https://www.amazon.com/Reckoning-Risk-Learning-Live-Uncertai... Here's an extract: https://imgur.com/zO4zkl4 https://imgur.com/zO4zkl4
rcdmd 8y ago
Reading some comments in this thread you might think physicians are unable to apply Bayesian statistics to medical care. As a physician myself, I'd encourage a more considered line of thinking. Clearly, physicians here failed to calculate an exact positive predictive value in the example. The question is does that inability affect their 1) medical care delivered and 2) communication with the patient and the patient's own informed decision-making. In speaking to 1-- there are many examples to choose from from probably any medical specialty but let's stick to breast cancer screening since that's the example from the article. USPSTF presents their recommendations[1]. I'd encourage anyone with interest to at least skim the rationale presented on the page below those recommendations. They very well consider prevalence as well as efficacy of specific tests given the presence of different risk factors in a patient (age, family history, etc). Importantly, those and many other screening guidelines are applied by primary care physicians who may not otherwise be able to calculate exacting probabilities. [1] https://www.uspreventiveservicestaskforce.org/Page/Document/RecommendationStatementFinal/breast-cancer-screening1 https://www.uspreventiveservicestaskforce.org/Page/Document/... In speaking to 2-- patient autonomy of course requires an appropriate understanding of the tests they receive, any risk to those tests and the benefits and harms of true positives, false negatives, etc. The associated frequencies, albeit with some degree of imprecision are avaialble by reference and I'd suspect they're memorized by most radiologists reading mammograms and the breast surgeons involved in tested positives-- even if they may not be able to calculate them. If the doctor doesn't have them memorized-- they should be available by referencing the relevant guideline.
- User23 8y ago
  I've never once in my life been able to give real informed consent to a physician, because I've never had one able to inform me quantitatively of the risks and benefits of accepting or refusing treatment. So like, I assume, everyone else I pretty much just have to take what the doctor says on blind faith. Since the vast majority of physicians are good people who actually want to help their patients, I'm not entirely uncomfortable with this, but it's still troubling given how many people die every year as a result of interventions or complications thereof.
  - 8y ago
deleted 8y ago
[deleted]
known 8y ago
Science, Statistics and Lies https://en.wikipedia.org/wiki/Lies,_damned_lies,_and_statistics https://en.wikipedia.org/wiki/Lies,_damned_lies,_and_statist...
fred_and_fred 8y ago
This topic comes up here periodically, and I've linked to these articles before, but it's probably worth doing so again for those interested. I am a physician who was also puzzled by these matters, and wound up inventing a simple diagram that I use as a mental "widget" to manipulate the ideas of the 2x2 table in my head. The full text articles can be found at: https://www.researchgate.net/profile/Kevin_Johnson40 https://www.researchgate.net/profile/Kevin_Johnson40 P.S. I'd love to have someone animate this in Javascript!
- fred_and_fred 8y ago
  The code for an R version of this is on Github at https://github.com/two-by-two https://github.com/two-by-two There is a limited demo there as well.

53 points by bladecatcher 8y ago

Sadly unsurprising. I remember reading a conversation on Twitter where a psychologist wrote openly "I don't use statistics because I have an analytical orientation".

tcj_phx 8y ago

I find it ironic that this article is posted on a website aimed at the mental health industry. The actual challenge faced by mental health practitioners is with the futility of their approach. I met a young man (early 20's) who'd recently arrived in town for the purposes of "recovery". Having learned something about people's problems with self-medicating I asked, "People usually know when their problems with substances started. When did it start for you?" He instantly said that when he was in third grade his teacher thought he was disruptive. Parents took him to the doctor, who prescribed medication. When he got to 9th grade he gave his parents an ultimatum: "if you don't take me off these drugs, I'm going to kill myself." His parents promptly took him off the drugs that didn't address the boredom he'd experienced in his suffocating 3rd grade classroom, and that's when he started self-medicating with whatever he could get his hands on. The DSM is the mental health industry's guide to help practitioners precisely diagnose symptoms. The problem is the industry frequently jumps to treatment without concern as to the cause of the patients' symptoms. Court-ordered treatment (aka "assisted outpatient treatment") is where the mental health industry decides to force a patient to take the drugs they think they need. I have the affidavits that were filed against my friend. They say, essentially, "Patient expects us to believe that her symptoms are related to consuming 2 bottles of liquor a day. She is clearly in need of court-ordered treatment because she does not believe that she has a chronic condition. We know that she has a chronic condition because she's had two previous orders for treatment. She took herself off our forced services 4 years ago, and here she is again." Before this went down, I'd decided that my friend's substance abuse problems were related to her having been adopted, exposed to meth amphetamine as a pre-teen, having "lost her future" at 16 years old when she was expelled for self-medicating with what is now approved as a breakthrough therapy for PTSD (MDMA), and having been injected with a prescription endocrine disruptor [0] with the black-box warning of "cortisol deficiency" at 18 or 19 years old. This "birth control" drug is known to make some women suicidally-depressed. Cortisol deficiency is now associated with psychosis. My friend allowed that maybe the injections took her from "drug abuser" to "drug addict". [0] https://en.wikipedia.org/wiki/Depo-Provera https://en.wikipedia.org/wiki/Depo-Provera In the world where mental health professionals address causes rather than symptoms and treat conservatively when the causes of a condition are unknown, those doctors would have recognized my friend's presentation as a form of substance-induced psychosis [1], and provided support to help her sober up. [1] https://en.wikipedia.org/wiki/Substance-induced_psychosis https://en.wikipedia.org/wiki/Substance-induced_psychosis Rather than provide rational treatment, they just force her to take "anti-psychotics" in the delusional belief that their drugs (rather than sobriety) are what allows my friend to be functional. At one point they thought she needed two different drugs. My friend explained her presentation as, "I'm sorry, these drugs make us slow". Every medical specialty has practices that aren't actually justified by the findings of science. I believe Psychiatry is the most important of the medical specialties, but the path "mainstream" practitioners has taken since the 1950's has been a mistake: most psychiatric drugs are just modern FDA-approved "patent medicines" [2] that don't actually address the causes of the patient's symptoms. While some people like their psych drugs, they'd probably do much better with more scientific approach to their complaints. [2] https://en.wikipedia.org/wiki/Patent_medicine https://en.wikipedia.org/wiki/Patent_medicine

CWSZ 8y ago
Rather than provide rational treatment, they just force her to take "anti-psychotics" in the delusional belief that their drugs (rather than sobriety) are what allows my friend to be functional. My best friend's father is a paranoid schizophrenic. Antipsychotic medication allows him to live a relatively normal life.
- tcj_phx 8y ago
  > My best friend's father is a paranoid schizophrenic. Antipsychotic medication allows him to live a relatively normal life. How do you know that it's the medication that allows him to "live a relatively normal life", and not the other factors? If you want to trade anecdotes... My other friend's father fell out of a tree and broke his back when she was a child (iirc). Some time later they decided he was a "schizophrenic". He's been on antipsychotics ever since. His condition has spiraled downward over the subsequent years. Last I heard he was full-on crazy. If the doctors had provided supportive treatment instead of suppressive, his life trajectory would have been totally different. My aunt's good friend had a "psychotic break" soon after her husband died unexpectedly. The friend was put on "anti-psychotics". She's now dying of liver failure, certainly as a consequence of her long-term drugging. The ugly truth about so-called "antipsychotic medication" is that the class is palliative rather than curative, and actually causes the deterioration it supposedly treats. Robert Whitaker makes the case that medications turns people's "episodes" into chronic conditions: https://www.madinamerica.com/2016/07/the-case-against-antipsychotics/ https://www.madinamerica.com/2016/07/the-case-against-antips... and https://www.madinamerica.com/2018/03/a-tale-of-two-studies/ https://www.madinamerica.com/2018/03/a-tale-of-two-studies/ and https://www.madinamerica.com/2017/09/thou-shall-not-criticize-our-drugs/ https://www.madinamerica.com/2017/09/thou-shall-not-criticiz... , for example.
  - CWSZ 8y ago
    I dislike exchanging anecdotes too. If you can point me to a treatment for schizophrenia tested in quality randomized trials with the success rate of antipsychotics (which still have a very poor success rate), please do.

mrnobody_67 8y ago

I always wonder the pros/cons of getting tested for no specific symptom... anxiety.stress.costs of false positives could be enormous. There's even a book & several articles out there arguing that getting tested for cancer might be a bad idea: https://www.theatlantic.com/health/archive/2015/06/should-I-get-tested-screened-for-cancer/397100/ https://www.theatlantic.com/health/archive/2015/06/should-I-...

iheartpotatoes 8y ago

BSEE and MSEE with 8 years of heavy math and the fact that I still have to read the solution slowly and not really get it scares the bejeezers out of me.

hugey010 8y ago
If you have any kind of test that seems relatively accurate (5% failure rate), 1 in every 20 tests will be identified as a positive candidate no matter the actual incidence rate. When applied over a huge population the number of positive candidates is going to also be huge. Since the incidence rate in this scenario is so low (1/1000) the odds of a true positive selected from the positive candidates is also very low.
tnecniv 8y ago
This is a little surprising to me. I was required to take a probability class for my BS in electrical engineering, and a version of this problem was a homework question.
- iheartpotatoes 8y ago
  Funny you should call this out: As an undergrad I got to choose between a prob & statistics path and a complex math / advance calc path. I took the complex math path. However, I was required to take a semester on engineering methods that touched on statistics, but that was part of one semester. As a grad student, I went down the numerical methods path for parallel computing. Not as many classes as undergrad, but still. College: RPI. Graduation year: '91.

DanBC 8y ago

Bit annoying that this article hat-tips Taleb, when Gerd Gigerenzer has been doing this better and for longer. This book is a good introduction, includes a lot of anecdote from different health care professionals across a range of different testing (including cancer and HIV), and mentions the research across different health care professionals. https://www.amazon.com/Reckoning-Risk-Learning-Live-Uncertainty/dp/0140297863/ https://www.amazon.com/Reckoning-Risk-Learning-Live-Uncertai... Here's an extract: https://imgur.com/zO4zkl4 https://imgur.com/zO4zkl4

rcdmd 8y ago

Reading some comments in this thread you might think physicians are unable to apply Bayesian statistics to medical care. As a physician myself, I'd encourage a more considered line of thinking. Clearly, physicians here failed to calculate an exact positive predictive value in the example. The question is does that inability affect their 1) medical care delivered and 2) communication with the patient and the patient's own informed decision-making. In speaking to 1-- there are many examples to choose from from probably any medical specialty but let's stick to breast cancer screening since that's the example from the article. USPSTF presents their recommendations[1]. I'd encourage anyone with interest to at least skim the rationale presented on the page below those recommendations. They very well consider prevalence as well as efficacy of specific tests given the presence of different risk factors in a patient (age, family history, etc). Importantly, those and many other screening guidelines are applied by primary care physicians who may not otherwise be able to calculate exacting probabilities. [1] https://www.uspreventiveservicestaskforce.org/Page/Document/RecommendationStatementFinal/breast-cancer-screening1 https://www.uspreventiveservicestaskforce.org/Page/Document/... In speaking to 2-- patient autonomy of course requires an appropriate understanding of the tests they receive, any risk to those tests and the benefits and harms of true positives, false negatives, etc. The associated frequencies, albeit with some degree of imprecision are avaialble by reference and I'd suspect they're memorized by most radiologists reading mammograms and the breast surgeons involved in tested positives-- even if they may not be able to calculate them. If the doctor doesn't have them memorized-- they should be available by referencing the relevant guideline.

User23 8y ago
I've never once in my life been able to give real informed consent to a physician, because I've never had one able to inform me quantitatively of the risks and benefits of accepting or refusing treatment. So like, I assume, everyone else I pretty much just have to take what the doctor says on blind faith. Since the vast majority of physicians are good people who actually want to help their patients, I'm not entirely uncomfortable with this, but it's still troubling given how many people die every year as a result of interventions or complications thereof.
- 8y ago

deleted 8y ago

[deleted]

known 8y ago

Science, Statistics and Lies https://en.wikipedia.org/wiki/Lies,_damned_lies,_and_statistics https://en.wikipedia.org/wiki/Lies,_damned_lies,_and_statist...

fred_and_fred 8y ago

This topic comes up here periodically, and I've linked to these articles before, but it's probably worth doing so again for those interested. I am a physician who was also puzzled by these matters, and wound up inventing a simple diagram that I use as a mental "widget" to manipulate the ideas of the 2x2 table in my head. The full text articles can be found at: https://www.researchgate.net/profile/Kevin_Johnson40 https://www.researchgate.net/profile/Kevin_Johnson40 P.S. I'd love to have someone animate this in Javascript!

fred_and_fred 8y ago
The code for an R version of this is on Github at https://github.com/two-by-two https://github.com/two-by-two There is a limited demo there as well.