29 ms·

> I am a software developer and if I write some piece of medical software with bugs that are so blatant in it, I'd see hell up to potentially being sued into ob

by phil21 28d ago

> I am a software developer and if I write some piece of medical software with bugs that are so blatant in it, I'd see hell up to potentially being sued into oblivion Based on watching the medical software field as a consumer (patient) and friends who are doctors, this is a fantasy. The quality of software in this field is abysmal and there seems to be almost no repercussions to those who develop or sell it. Which is precisely why this sort of thing can be rolled out without much fear by those pushing it.

voxl 28d ago
The software is heavily regulated for medical devices. Saying an MRI machine has bad software seems highly unlikely to me. This is of course different from Epic, but even then as a patient MyChart is really not that bad
febusravenga 28d ago
That's besides point, the point is that classical software bugs are contained and we have process and understanding allowing us to discover them, triqge and fix. For AI based sw you can guess and try another prompt.
- kristiandupont 28d ago
  Why is that "the point"? You are the second person in this thread to insist on owning it. I get why it's problematic, obviously, but if it produces statistically better results (which I have no idea of), I don't think it's right to just write it off because of this.
  - WelkinFolk 28d ago
    Because you cannot hold people accountable, since it isn't "people". We humans are emotional creatures, and we will always prefer systems where someone can be held accountable for mistakes. The developer, the doctor. Whoever. I do not oppose AI integration; I'm not a Luddite. But having a "move fast, who cares if a couple die" isn't the way to go with sensitive fields, like the medical field. I suppose we will come up with proper responsibility-hierarchies and guardrails around AI, but until then, people have a right to complain about the lack of them.
intended 28d ago
Isn’t medical software highly regulated ? You sure there is no fear ?
fragmede 28d ago
The case of Therac-25 is required reading for a computer science ethics course. (How requisite a CS ethics course is a different story entirely though.) It's not a fantasy for more regulated medical devices eg pacemakers.

by phil21 28d ago

voxl 28d ago

The software is heavily regulated for medical devices. Saying an MRI machine has bad software seems highly unlikely to me. This is of course different from Epic, but even then as a patient MyChart is really not that bad

febusravenga 28d ago

That's besides point, the point is that classical software bugs are contained and we have process and understanding allowing us to discover them, triqge and fix. For AI based sw you can guess and try another prompt.

kristiandupont 28d ago
Why is that "the point"? You are the second person in this thread to insist on owning it. I get why it's problematic, obviously, but if it produces statistically better results (which I have no idea of), I don't think it's right to just write it off because of this.
- WelkinFolk 28d ago
  Because you cannot hold people accountable, since it isn't "people". We humans are emotional creatures, and we will always prefer systems where someone can be held accountable for mistakes. The developer, the doctor. Whoever. I do not oppose AI integration; I'm not a Luddite. But having a "move fast, who cares if a couple die" isn't the way to go with sensitive fields, like the medical field. I suppose we will come up with proper responsibility-hierarchies and guardrails around AI, but until then, people have a right to complain about the lack of them.

intended 28d ago

Isn’t medical software highly regulated ? You sure there is no fear ?

fragmede 28d ago

The case of Therac-25 is required reading for a computer science ethics course. (How requisite a CS ethics course is a different story entirely though.) It's not a fantasy for more regulated medical devices eg pacemakers.