5 ms·

Just the other day there were news about how many mistakes the AI-assisted journaling caused – wrong names, wrong diagnoses, typos (buksmärta -> kuksmärta which

Just the other day there were news about how many mistakes the AI-assisted journaling caused – wrong names, wrong diagnoses, typos (buksmärta -> kuksmärta which is hilarious but serious). Some of it is surely teething problems, but unless there is a robust check upon implementation it might just add another layer of inefficient new public management make-work to the system. https://sverigesradio.se/artikel/ai-journaler-i-sjukvarden-kritiseras-buk-blev-kuk https://sverigesradio.se/artikel/ai-journaler-i-sjukvarden-k...

Angostura 2y ago
It feels to me like an autopilot problem in the making. "This thing means that you don't have to keep your eyes on the road - but please ensure you keep your eyes on the road, in case of errors"
- kohbo 2y ago
  I found it interesting that your mind went to Tesla's autopilot. My mind went to operating airplanes. Most newer small planes have some form of GPS but you're technically not supposed to use instrument navigation until your certified to do so. I haven't met a single pilot that didn't do so, though. Anyway, it creates the very problem you mentioned but just replace "road" with "outside the cockpit".
  - sco1 2y ago
    > but you're technically not supposed to use instrument navigation until your certified to do so What do you mean by this? Not having an IFR rating does not mean you're not allowed to use the navigation aids or the plane's autopilot.
  - ceejayoz 2y ago
    > you're technically not supposed to use instrument navigation until your certified to do so You can use them all you like. You just can't fly in conditions where you have to use them.
- pixl97 2y ago
  The issue is, if you have any kind of rare condition, it already is this way. Much like the entire white side of a semi being presented to you across the road is a rare condition for autopilot, a huge number of 'rare' diseases already present problems for humans leading doctors and their staff to make errors by assuming the most likely condition. There is some saying like "It's probably a horse and not a zebra", but when it comes to hospitals zebras and even unicorns do show up, especially in the cases with recurring problems.
elitan 2y ago
I wouldn't be surprised if Region Blekinge were using something much worse and much more expensive than Whisper for their transcription. I've been transcribing A LOT of SR (Swedish Radio) shows as part of https://nyheter.sh/ https://nyheter.sh/, and Whisper (self-hosted) has been very accurate.
- popinman322 2y ago
  Tangent here: really? I've found base Whisper has concerning error rates for non-US English accents; I imagine the same is true for other languages with a large regional mode to the source dataset. Whisper + an LLM can recover some of the gaps by filling in contextually plausible bits, but then it's not a transcript and may contain hallucinations. There are alternatives that share Whisper internal states with an LLM to improve ASR, as well as approaches that sample N-best hypotheses from Whisper and fine-tune an LLM to distill the hypotheses into a single output. Haven't looked too much into these yet given how expensive each component is to run independently.
  - vidyava 2y ago
    I've found that WhisperX with the medium model has been amazing at subtitling shows containing English dialects (British, Scottish, Australian, New Zealand-ish). It not only nails all the normal speech, but even gets the names and completely made up slang words. Interestingly you can tell it was trained from source material with dialects because it subtitles their particular spelling; so someone American will say color, and someone British will say colour. I can't speak to how it performs outside of production quality audio, but in the hundreds of hours of subtitles that I've generated I don't think I've seen a single error.
    - rhdunn 2y ago
      IIUC, it's trained on LibriVox audio mainly, along with a few other sources. I'm not sure how it is handling spelling as the spelling will depend on the source content being read, unless the source text has been processed/edited to align with the dialect.
  - 2y ago
ThePowerOfFuet 2y ago
> typos (buksmärta -> kuksmärta which is hilarious but serious) To save people looking it up, that one-char difference changes "abdominal pain" into "cock pain". Wow.
- hulitu 2y ago
  Well, they are close enough to each other, aren't they ? /s That is the main problem with the AI: it is close enough, but never there.
williamcotton 2y ago
Reading and doing minor edits is much less of a cognitive load than writing. The article does not suggest that doctors should blindly trust the SOAP note created by the tool in question.
- ceejayoz 2y ago
  > The article does not suggest that doctors should blindly trust the SOAP note created by the tool in question. But that's what will inevitably happen at some point, when they get to the point of only rarely making big dangerous mistakes.
  - MichaelZuo 2y ago
    Isn't the vast majority of the world using computers without ECC memory already blindly trusting there are no bit flips causing silent corruption?
  - snoman 2y ago
    > …when they get to the point of only rarely making big dangerous mistakes. Are you intentionally rubbing FUD on that or am I mis-reading you? I don’t think we need to wait to rely on technologies until they’ve achieved perfection - just when their mistakes are less frequent, less dangerous, or more predictable than human mistakes for the same task.
    - ceejayoz 2y ago
      I'm saying getting close to perfection is truly dangerous territory, because everyone gets very complacent at that point. As a concrete example: https://www.cbsnews.com/news/pilots-fall-asleep-mid-flight-150-on-board-batik-air-indonesia/ https://www.cbsnews.com/news/pilots-fall-asleep-mid-flight-1...
      - danielmarkbruce 2y ago
        We are already there with humans. Most people take a doctor at their word and don't bother to get a second opinion.
deleted 2y ago
[deleted]
_wire_ 2y ago
> Some of it is surely teething problems... Just gonna adjust the temperature of your baby here, ok, now he should grow up just fine.
chollida1 2y ago
This seems very similar to self driving cars. At the beginning they will be worse than humans and cause deaths that humans would have prevented while at the same time probably saving lives where a human would make a mistake. But not far down the road they'll become much better than humans even if they do occasionally make a mistake and cause a death that a human wouldn't' have they'll save far more lives due to them not making the mistakes that humans do.
- b112 2y ago
  But not far down the road they'll become much better than humans While I think you're correct, there is no proof this will ever be achieved. It very well may not be possible along our current path. It may take a 100 years, 1000 to get there. And yes it could take only 20 more. But to state this as a certainty? No.
  - chollida1 2y ago
    I think its basic certainty that computers will be better at transcribing notes than humans basically now and will continue to get better. I mean we're basically there now, I trust spell check and grammar check more than myself. That's what we're talking about here. IMHO computers are already better at that, what possibly makes you think this won't happen?
    - b112 2y ago
      My reply was to the parent post, which started: This seems very similar to self driving cars. And continued discussing self driving cars. Regardless, what I said stands.
      - chollida1 2y ago
        ah, then I mis-interpreted. My fault, appreciate you letting me know.
masto 2y ago
This would be a major concern for me. The one time I used AI meeting notes, some important details were wrong. And beyond that, the notes were just terrible. A literal transcription can be useful. A summary of the substance of the meeting can be useful. This was neither. A human would know that a tangent talking about the weather is not important, but AI notes are just as likely to fill the document with "Chris mentioned it had rained yesterday but he was hoping to cook hamburgers when the sun comes out. Alice and Bob expressed opinions about side dishes, with the consensus being that fries are more appropriate than potato salad." as it was to miss a nuanced point that a human would have recorded because they understood the purpose of the meeting. And then it'd give me an action item to buy corn.
fencepost 2y ago
This is a situation where running within the context of the EMR and having access to the existing chart data is likely to make a lot of difference. My bigger concern is that there are a lot of different things being lumped together under "AI", and this is going to hit a bunch of different areas of machine learning.

Angostura 2y ago

It feels to me like an autopilot problem in the making. "This thing means that you don't have to keep your eyes on the road - but please ensure you keep your eyes on the road, in case of errors"

kohbo 2y ago
I found it interesting that your mind went to Tesla's autopilot. My mind went to operating airplanes. Most newer small planes have some form of GPS but you're technically not supposed to use instrument navigation until your certified to do so. I haven't met a single pilot that didn't do so, though. Anyway, it creates the very problem you mentioned but just replace "road" with "outside the cockpit".
- sco1 2y ago
  > but you're technically not supposed to use instrument navigation until your certified to do so What do you mean by this? Not having an IFR rating does not mean you're not allowed to use the navigation aids or the plane's autopilot.
- ceejayoz 2y ago
  > you're technically not supposed to use instrument navigation until your certified to do so You can use them all you like. You just can't fly in conditions where you have to use them.
pixl97 2y ago
The issue is, if you have any kind of rare condition, it already is this way. Much like the entire white side of a semi being presented to you across the road is a rare condition for autopilot, a huge number of 'rare' diseases already present problems for humans leading doctors and their staff to make errors by assuming the most likely condition. There is some saying like "It's probably a horse and not a zebra", but when it comes to hospitals zebras and even unicorns do show up, especially in the cases with recurring problems.

elitan 2y ago

I wouldn't be surprised if Region Blekinge were using something much worse and much more expensive than Whisper for their transcription. I've been transcribing A LOT of SR (Swedish Radio) shows as part of https://nyheter.sh/ https://nyheter.sh/, and Whisper (self-hosted) has been very accurate.

popinman322 2y ago
Tangent here: really? I've found base Whisper has concerning error rates for non-US English accents; I imagine the same is true for other languages with a large regional mode to the source dataset. Whisper + an LLM can recover some of the gaps by filling in contextually plausible bits, but then it's not a transcript and may contain hallucinations. There are alternatives that share Whisper internal states with an LLM to improve ASR, as well as approaches that sample N-best hypotheses from Whisper and fine-tune an LLM to distill the hypotheses into a single output. Haven't looked too much into these yet given how expensive each component is to run independently.
- vidyava 2y ago
  I've found that WhisperX with the medium model has been amazing at subtitling shows containing English dialects (British, Scottish, Australian, New Zealand-ish). It not only nails all the normal speech, but even gets the names and completely made up slang words. Interestingly you can tell it was trained from source material with dialects because it subtitles their particular spelling; so someone American will say color, and someone British will say colour. I can't speak to how it performs outside of production quality audio, but in the hundreds of hours of subtitles that I've generated I don't think I've seen a single error.
  - rhdunn 2y ago
    IIUC, it's trained on LibriVox audio mainly, along with a few other sources. I'm not sure how it is handling spelling as the spelling will depend on the source content being read, unless the source text has been processed/edited to align with the dialect.
- 2y ago

ThePowerOfFuet 2y ago

> typos (buksmärta -> kuksmärta which is hilarious but serious) To save people looking it up, that one-char difference changes "abdominal pain" into "cock pain". Wow.

hulitu 2y ago
Well, they are close enough to each other, aren't they ? /s That is the main problem with the AI: it is close enough, but never there.

williamcotton 2y ago

Reading and doing minor edits is much less of a cognitive load than writing. The article does not suggest that doctors should blindly trust the SOAP note created by the tool in question.

ceejayoz 2y ago
> The article does not suggest that doctors should blindly trust the SOAP note created by the tool in question. But that's what will inevitably happen at some point, when they get to the point of only rarely making big dangerous mistakes.
- MichaelZuo 2y ago
  Isn't the vast majority of the world using computers without ECC memory already blindly trusting there are no bit flips causing silent corruption?
- snoman 2y ago
  > …when they get to the point of only rarely making big dangerous mistakes. Are you intentionally rubbing FUD on that or am I mis-reading you? I don’t think we need to wait to rely on technologies until they’ve achieved perfection - just when their mistakes are less frequent, less dangerous, or more predictable than human mistakes for the same task.
  - ceejayoz 2y ago
    I'm saying getting close to perfection is truly dangerous territory, because everyone gets very complacent at that point. As a concrete example: https://www.cbsnews.com/news/pilots-fall-asleep-mid-flight-150-on-board-batik-air-indonesia/ https://www.cbsnews.com/news/pilots-fall-asleep-mid-flight-1...
    - danielmarkbruce 2y ago
      We are already there with humans. Most people take a doctor at their word and don't bother to get a second opinion.

deleted 2y ago

[deleted]

_wire_ 2y ago

> Some of it is surely teething problems... Just gonna adjust the temperature of your baby here, ok, now he should grow up just fine.

chollida1 2y ago

This seems very similar to self driving cars. At the beginning they will be worse than humans and cause deaths that humans would have prevented while at the same time probably saving lives where a human would make a mistake. But not far down the road they'll become much better than humans even if they do occasionally make a mistake and cause a death that a human wouldn't' have they'll save far more lives due to them not making the mistakes that humans do.

b112 2y ago
But not far down the road they'll become much better than humans While I think you're correct, there is no proof this will ever be achieved. It very well may not be possible along our current path. It may take a 100 years, 1000 to get there. And yes it could take only 20 more. But to state this as a certainty? No.
- chollida1 2y ago
  I think its basic certainty that computers will be better at transcribing notes than humans basically now and will continue to get better. I mean we're basically there now, I trust spell check and grammar check more than myself. That's what we're talking about here. IMHO computers are already better at that, what possibly makes you think this won't happen?
  - b112 2y ago
    My reply was to the parent post, which started: This seems very similar to self driving cars. And continued discussing self driving cars. Regardless, what I said stands.
    - chollida1 2y ago
      ah, then I mis-interpreted. My fault, appreciate you letting me know.

masto 2y ago

This would be a major concern for me. The one time I used AI meeting notes, some important details were wrong. And beyond that, the notes were just terrible. A literal transcription can be useful. A summary of the substance of the meeting can be useful. This was neither. A human would know that a tangent talking about the weather is not important, but AI notes are just as likely to fill the document with "Chris mentioned it had rained yesterday but he was hoping to cook hamburgers when the sun comes out. Alice and Bob expressed opinions about side dishes, with the consensus being that fries are more appropriate than potato salad." as it was to miss a nuanced point that a human would have recorded because they understood the purpose of the meeting. And then it'd give me an action item to buy corn.

fencepost 2y ago

This is a situation where running within the context of the EMR and having access to the existing chart data is likely to make a lot of difference. My bigger concern is that there are a lot of different things being lumped together under "AI", and this is going to hit a bunch of different areas of machine learning.