5 ms·

The other side is that I have witnessed doctors making mistakes personally on medical files. They are overworked and only see you for 15 minutes and in a rush t

by pizzly 28d ago

The other side is that I have witnessed doctors making mistakes personally on medical files. They are overworked and only see you for 15 minutes and in a rush to kick you out and so of course they make errors. It would be interesting to compare whether transcription software or humans make the most mistakes. Guessing the answer will be very nuanced such as transcription software failing for accents not trained on, how over worked the clinic is etc.

tdeck 28d ago
There's another option, which is to have the consultation recorded and then transcribed by a third person, or at least have the transcript reviewed while listening to the recording. I think for medical situations it's more than reasonable to put in the labor to ensure correctness.
- selestify 28d ago
  Cost of medical care is already sky high (in the US). Are you sure you want to introduce even more administrative overhead?
  - tdeck 28d ago
    How were people doing this a few years ago before generative AI? Did the cost of care go down significantly and I just missed it?
- bpodgursky 28d ago
  Think about what this literally means. There are a million doctors in the US. Doctors spend their day consulting patients, so, you need a million transcription reviewers. Let's say you pay each of them $50k/yr (aka rock bottom US entry-level wage). That's $50 billion a year, and a million people pulled out of the labor force. Is that worth it? Is that really the best thing you can do with $50 billion dollars? In real life, tradeoffs exist.
  - nradov 28d ago
    There are already a lot of people doing manual review and editing of medical transcription speech recognition output. That has been the status quo for decades. A lot of those workers are offshore in India and Philippines. Overall transcription work is slowly declining as more physicians switch to direct EHR template data entry and AI scribes.
- nradov 28d ago
  Medical transcription isn't the same thing as medical scribing. Transcription has been done primarily by speech recognition software with human review for decades. The output is just a text chart note, which isn't particularly useful because none of the clinical findings are coded. The coding can either be done up front by a human or AI scribe, or as a second step after transcription. In any case the human attending clinician is legally accountable for reviewing and approving the output before it's officially added to the patient chart.
altmanaltman 28d ago
> Human medical scribes manually document encounters in real-time and, in randomized trials, are more than four times as likely to produce notes physicians rate as ‘accurate’ compared with standard self-documentation4. Automated speech-recognition dictation systems generally have higher error rates—typically 7–11%—owing to the complexity of medical jargon and accent variability5 https://pmc.ncbi.nlm.nih.gov/articles/PMC12460601/ https://pmc.ncbi.nlm.nih.gov/articles/PMC12460601/
- romaniitedomum 28d ago
  It used to be common in some places to outsource medical transcription to India (obviously not real-time recording). I spent time in India years ago working with a company in Mumbai (not medical transcription) and there were stories in Indian newspapers about mistakes being made due to the transcribers having no medical knowledge at all. One instance that I've never forgotten is the one where the Doctor had diagnosed "phlebitis" in the patient's leg, and the transcriber had written "a flea bite his leg".
  - altmanaltman 28d ago
    Yes annecdotal evidence is a perfect counterfactual to that.
autumnstwilight 28d ago
How overworked would a doctor need to be to hallucinate you told them about microdosing psillocybin when you never said anything of the sort? (And how easy is it to then convince the system you're not a recreational drug user when it's in your medical records you admitted to drug use? Does this change if you're from a background more likely to be stereotyped as drug users?) The humans are probably making mistakes like writing 100mg of something when they meant 10mg, which can also have dangerous consequences, but to an extent it's known this happens and there are processes to catch it. The type of mistakes and how much trouble and distress they cause matters as much as the number.
- srveale 28d ago
  I recommend reading table 3 in this paper: https://pmc.ncbi.nlm.nih.gov/articles/PMC7284300/ https://pmc.ncbi.nlm.nih.gov/articles/PMC7284300/ "It was stated in notes that I had lung cancer. I do not and never have had lung cancer." "I did receive a referral for physical therapy, it was for the wrong body part" "Doctor reported that I did not claim to have pain in my hand. I am a pianist and I went specifically because pain was in my hand." "I have been complaining of difficulty breathing [for over 3 mo]... notes saying my breathing is normal" And plenty more
- cryzinger 28d ago
  To this day we have no idea why or how, but years ago someone at a chain of walk-in clinics once put a note in my wife's file indicating that she was a habitual cocaine user. She had no idea until she went in one time and they were like "So, are you still using cocaine?" (I do think this is extraordinarily rare compared to how often AI hallucinations happen, but it's such a bizarre story that I have to mention it.)
- martin- 27d ago
  > How overworked would a doctor need to be to hallucinate you told them about microdosing psillocybin when you never said anything of the sort From personal experience I can tell you that it does indeed happen. Not specifically micro-dosing psilocybin of course, but of a similar nature, and completely made up. Not sure how it happens.
nozzlegear 28d ago
> They are overworked and only see you for 15 minutes and in a rush to kick you out and so of course they make errors. Damn, my doctor is always happy to give me 45 minutes of her time or more. Sometimes I feel like I'm taking up too much of her time chatting her ear off, but she's never in a rush to get me out of the door. Perks of small town living?

by pizzly 28d ago

tdeck 28d ago

There's another option, which is to have the consultation recorded and then transcribed by a third person, or at least have the transcript reviewed while listening to the recording. I think for medical situations it's more than reasonable to put in the labor to ensure correctness.

selestify 28d ago
Cost of medical care is already sky high (in the US). Are you sure you want to introduce even more administrative overhead?
- tdeck 28d ago
  How were people doing this a few years ago before generative AI? Did the cost of care go down significantly and I just missed it?
bpodgursky 28d ago
Think about what this literally means. There are a million doctors in the US. Doctors spend their day consulting patients, so, you need a million transcription reviewers. Let's say you pay each of them $50k/yr (aka rock bottom US entry-level wage). That's $50 billion a year, and a million people pulled out of the labor force. Is that worth it? Is that really the best thing you can do with $50 billion dollars? In real life, tradeoffs exist.
- nradov 28d ago
  There are already a lot of people doing manual review and editing of medical transcription speech recognition output. That has been the status quo for decades. A lot of those workers are offshore in India and Philippines. Overall transcription work is slowly declining as more physicians switch to direct EHR template data entry and AI scribes.
nradov 28d ago
Medical transcription isn't the same thing as medical scribing. Transcription has been done primarily by speech recognition software with human review for decades. The output is just a text chart note, which isn't particularly useful because none of the clinical findings are coded. The coding can either be done up front by a human or AI scribe, or as a second step after transcription. In any case the human attending clinician is legally accountable for reviewing and approving the output before it's officially added to the patient chart.

altmanaltman 28d ago

> Human medical scribes manually document encounters in real-time and, in randomized trials, are more than four times as likely to produce notes physicians rate as ‘accurate’ compared with standard self-documentation4. Automated speech-recognition dictation systems generally have higher error rates—typically 7–11%—owing to the complexity of medical jargon and accent variability5 https://pmc.ncbi.nlm.nih.gov/articles/PMC12460601/ https://pmc.ncbi.nlm.nih.gov/articles/PMC12460601/

romaniitedomum 28d ago
It used to be common in some places to outsource medical transcription to India (obviously not real-time recording). I spent time in India years ago working with a company in Mumbai (not medical transcription) and there were stories in Indian newspapers about mistakes being made due to the transcribers having no medical knowledge at all. One instance that I've never forgotten is the one where the Doctor had diagnosed "phlebitis" in the patient's leg, and the transcriber had written "a flea bite his leg".
- altmanaltman 28d ago
  Yes annecdotal evidence is a perfect counterfactual to that.

autumnstwilight 28d ago

How overworked would a doctor need to be to hallucinate you told them about microdosing psillocybin when you never said anything of the sort? (And how easy is it to then convince the system you're not a recreational drug user when it's in your medical records you admitted to drug use? Does this change if you're from a background more likely to be stereotyped as drug users?) The humans are probably making mistakes like writing 100mg of something when they meant 10mg, which can also have dangerous consequences, but to an extent it's known this happens and there are processes to catch it. The type of mistakes and how much trouble and distress they cause matters as much as the number.

srveale 28d ago
I recommend reading table 3 in this paper: https://pmc.ncbi.nlm.nih.gov/articles/PMC7284300/ https://pmc.ncbi.nlm.nih.gov/articles/PMC7284300/ "It was stated in notes that I had lung cancer. I do not and never have had lung cancer." "I did receive a referral for physical therapy, it was for the wrong body part" "Doctor reported that I did not claim to have pain in my hand. I am a pianist and I went specifically because pain was in my hand." "I have been complaining of difficulty breathing [for over 3 mo]... notes saying my breathing is normal" And plenty more
cryzinger 28d ago
To this day we have no idea why or how, but years ago someone at a chain of walk-in clinics once put a note in my wife's file indicating that she was a habitual cocaine user. She had no idea until she went in one time and they were like "So, are you still using cocaine?" (I do think this is extraordinarily rare compared to how often AI hallucinations happen, but it's such a bizarre story that I have to mention it.)
martin- 27d ago
> How overworked would a doctor need to be to hallucinate you told them about microdosing psillocybin when you never said anything of the sort From personal experience I can tell you that it does indeed happen. Not specifically micro-dosing psilocybin of course, but of a similar nature, and completely made up. Not sure how it happens.

nozzlegear 28d ago

> They are overworked and only see you for 15 minutes and in a rush to kick you out and so of course they make errors. Damn, my doctor is always happy to give me 45 minutes of her time or more. Sometimes I feel like I'm taking up too much of her time chatting her ear off, but she's never in a rush to get me out of the door. Perks of small town living?