10 ms·

Being close relative for several med and care workers we have discussed it a lot and consensus is that it really depends. For example relying on LLM summaries s

Being close relative for several med and care workers we have discussed it a lot and consensus is that it really depends. For example relying on LLM summaries sounds great until it doesn't. It doesn't matter whether you misunderstand LLM summary or LLM "misunderstands" you – there are real risks involved, and you wouldn't want them to weigh on your conscience if they were to materialize. Relying on LLM to summarize things for you has one more issue. To outsiders, this seems like a tedious process, but is actually very important part of the thought process. Wording your thoughts and writing these down helps people to discover new aspects of the problem. It's how people learn. At the moment consensus is that it must not be banned, but also not mandated in any way - people must take responsibility, and they must be able to decide for themselves where and when the LLM use is justified and where it is not.

red75prime 2mo ago
> where and when the LLM use is justified and where it is not while being bombarded with articles like "AI makes things worse", "AI consumes all the water" and the like
deleted 2mo ago
[deleted]
adrianN 2mo ago
I don’t think that it is possible to both allow the use of LLM and not mandate them in modern metric driven work places. Either you ban them or you force people to use them for game theoretic reasons: they are slower than their peers and quality of the work is harder to measure than quantity. All you achieve is shifting the blame to the employees if the LLM messes up. Come to think of it, that probably is a highly desirable outcome for the decision makers, so perhaps that will actually be the policy that becomes universally adopted.
- KronisLV 2mo ago
  > I don’t think that it is possible to both allow the use of LLM and not mandate them in modern metric driven work places. What I’d personally be most concerned about would be the risk of bad models instead of SOTA being used which would be especially error and hallucination prone. If you’re gonna do it, do it right. Otherwise don’t bother at all.
- squigz 2mo ago
  > All you achieve is shifting the blame to the employees if the LLM messes up. How is this not a good thing for everyone?
  - HouseofCardsHOC 2mo ago
    [flagged]
grayhatter 2mo ago
> It doesn't matter whether you misunderstand LLM summary or LLM "misunderstands" you – there are real risks involved, and you wouldn't want them to weigh on your conscience if they were to materialize. "fun" fact, a number (some? many? all?) of the LLM speech to text models will editorialize or hallucinate words for you, to make your speech fit the pattern it expects. I don't think it's appropriate to use a model that can editorialize in a medical transcription setting. I feel that crosses a huge number of ethical lines. > At the moment consensus is that it must not be banned, but also not mandated in any way - people must take responsibility, and they must be able to decide for themselves where and when the LLM use is justified and where it is not. I have a really hard time understanding how supposed experts in their field, have been taken in so perfectly and so completely. You're a trained and certified medical/healthcare worker, responsible for the health, safety and well being of other humans, and yet willingly turn over parts of the process that exists to protect the safety and health of others, to a machine that you know can and will make mistakes you wouldn't or even couldn't ever reasonably make. You're supposed to make sure it works and is safe for everyone you're responsible for before you start using it... but then, I prefer healthcare that doesn't move fast, and break humans. I'm sure someone will try to play devil's advocate and suggest it's a good thing to ration care, and that you have to acknowledge that you can't save everybody, faster is better after all But no, I'm angry enough about it that if a licensed person suggests that, I'll gladly complain to your licensing board (rhetorically speaking, I'm not gonna try to doxx anyone). You can't ethically argue for rationing care, and letting a machine known to make errors in ethics or care decisions, that you have been trained not to make. (Advocating for a reduction in the standard of care in order to increase profits.) (Yes, you're correct, if you're in a state of triage, you're required to, and in those cases, using an imperfect machine is possibly preferable, and probably ethical, but still objectionable. I currently refuse to believe that the US is in a state of emergency, but perhaps that's where I'm wrong) Part of your licensure you promised to prioritize patient care above personal gain. You can't then make decisions to experiment with systems you cant prove are safe. I can understand in when SWEs ignore the best interests of the humans they're working for. But Doctors and Nurses? I guess I expected better from my former colleagues?
- phoghed 2mo ago
  > I have a really hard time understanding how supposed experts in their field, have been taken in so perfectly and so completely. Yes, surely it’s not you. It’s all the experts that are wrong and being hoodwinked. Why can’t they just see what you can?

red75prime 2mo ago

> where and when the LLM use is justified and where it is not while being bombarded with articles like "AI makes things worse", "AI consumes all the water" and the like

deleted 2mo ago

[deleted]

adrianN 2mo ago

I don’t think that it is possible to both allow the use of LLM and not mandate them in modern metric driven work places. Either you ban them or you force people to use them for game theoretic reasons: they are slower than their peers and quality of the work is harder to measure than quantity. All you achieve is shifting the blame to the employees if the LLM messes up. Come to think of it, that probably is a highly desirable outcome for the decision makers, so perhaps that will actually be the policy that becomes universally adopted.

KronisLV 2mo ago
> I don’t think that it is possible to both allow the use of LLM and not mandate them in modern metric driven work places. What I’d personally be most concerned about would be the risk of bad models instead of SOTA being used which would be especially error and hallucination prone. If you’re gonna do it, do it right. Otherwise don’t bother at all.
squigz 2mo ago
> All you achieve is shifting the blame to the employees if the LLM messes up. How is this not a good thing for everyone?
- HouseofCardsHOC 2mo ago
  [flagged]

grayhatter 2mo ago

> It doesn't matter whether you misunderstand LLM summary or LLM "misunderstands" you – there are real risks involved, and you wouldn't want them to weigh on your conscience if they were to materialize. "fun" fact, a number (some? many? all?) of the LLM speech to text models will editorialize or hallucinate words for you, to make your speech fit the pattern it expects. I don't think it's appropriate to use a model that can editorialize in a medical transcription setting. I feel that crosses a huge number of ethical lines. > At the moment consensus is that it must not be banned, but also not mandated in any way - people must take responsibility, and they must be able to decide for themselves where and when the LLM use is justified and where it is not. I have a really hard time understanding how supposed experts in their field, have been taken in so perfectly and so completely. You're a trained and certified medical/healthcare worker, responsible for the health, safety and well being of other humans, and yet willingly turn over parts of the process that exists to protect the safety and health of others, to a machine that you know can and will make mistakes you wouldn't or even couldn't ever reasonably make. You're supposed to make sure it works and is safe for everyone you're responsible for before you start using it... but then, I prefer healthcare that doesn't move fast, and break humans. I'm sure someone will try to play devil's advocate and suggest it's a good thing to ration care, and that you have to acknowledge that you can't save everybody, faster is better after all But no, I'm angry enough about it that if a licensed person suggests that, I'll gladly complain to your licensing board (rhetorically speaking, I'm not gonna try to doxx anyone). You can't ethically argue for rationing care, and letting a machine known to make errors in ethics or care decisions, that you have been trained not to make. (Advocating for a reduction in the standard of care in order to increase profits.) (Yes, you're correct, if you're in a state of triage, you're required to, and in those cases, using an imperfect machine is possibly preferable, and probably ethical, but still objectionable. I currently refuse to believe that the US is in a state of emergency, but perhaps that's where I'm wrong) Part of your licensure you promised to prioritize patient care above personal gain. You can't then make decisions to experiment with systems you cant prove are safe. I can understand in when SWEs ignore the best interests of the humans they're working for. But Doctors and Nurses? I guess I expected better from my former colleagues?

phoghed 2mo ago
> I have a really hard time understanding how supposed experts in their field, have been taken in so perfectly and so completely. Yes, surely it’s not you. It’s all the experts that are wrong and being hoodwinked. Why can’t they just see what you can?