5 ms·

I think this creates an interesting discussion on the part of the designers and programmers. Did they understand that what they were building was going to incre

by pimmen 7y ago

I think this creates an interesting discussion on the part of the designers and programmers. Did they understand that what they were building was going to increase opioid dependence? Did they ask about it? And, the most interesting question of all: If they found out that was the case did they try to justify building it to themselves in some way? Full disclaimer: I have built software that I had ethical reservations about myself. I am in no way a saint, I don't claim to know what I would've done in the same situation but I think it's for the benefit of software development to acknowledge that we do make ethical decisions when we build things and that the solution to something as complicated and diverse as ethics is not to just ignore it.

nkrisc 7y ago
To be fair to the developers and designers: how exactly would they know that the inclusion of "opioid prescription" as an item in that list would help fuel a an opioid epidemic? Hindsight is 20/20, but would you expect them at the time to think, "If we include this as an option in this list, it will contribute to a nationwide epidemic of opioid abuse and countless deaths"? Honestly, I think it's a stretch, especially if they weren't aware of a deal with opioid manufacturers being the reason for it in the first place. It would be one thing of some product manager came to them and said, "since a large opioid manufacturer has paid us a lot of money, we're going to implement this feature in order to increase unnecessary opioid prescriptions" but I would be surprised if that happened. "We're going to implement this feature to help doctors pick the right treatment for a patient's pain." Someone here is evil and deserves what's coming, but I don't know that it necessarily would have been readily obvious to those implementing it. In fact, it would likely have been preferable to conceal the true nature of it lest any of them have any moral qualms or objections. I'm not trying to negate your question, either. I think it's a good question to ask and consider.
- mehlmao 7y ago
  This "feature" was added to the software in 2016, well into the epidemic.
- ryandrake 7y ago
  I’ve never been a big fan of the “I don’t know what my software does—I just move JSON around” excuse. As professionals, we need to be aware of the intended use of the things we make, and that includes weighing the ethical implications. We also need a professional ethical vow with teeth like the Hippocratic Oath that allows tech professionals to push back against demands to write unethical software, but that’s a topic for another thread.
  - nkrisc 7y ago
    I don't disagree. But my point was that it's not always clear what the implications are. Sometimes it's more obvious, sometime it's not. We also don't know what the "intended" use of this feature that was presented to the people who made it was. They could very well have been deceived as to it's true nature. This could very easily have been presented and sold to a team in a way that does not make it immediately seem like something evil.
    - mcguire 7y ago
      The bottom line is that developers and designers are not professionals, who would have to accept responsibility for their products. It is in the best interests of developers and designers not to be professionals and not to accept responsibility in cases like this. A standard argument for this state of affairs is that it is in the best interest of society to allow developers and designers this freedom from responsibility, because of all the shiny new products. In which case you have to accept incidents like this---you can't make an omelette without breaking a few eggs.
- jfarmer 7y ago
  > If we include this as an option in this list, it will contribute to a nationwide epidemic of opioid abuse and countless deaths? Give. me. a. break! At a minimum they should ask, "What are the potential downsides? How extreme are they? How can we determine how likely those potential downsides are?" I'd expect every junior PM to ask _that_ series of questions about any product decision in any market. There's a reason clinical trials have protocols for early termination. Not understanding that early termination is a thing when building a social app is one thing, but they're building EMR software!
  - mywittyname 7y ago
    Must be nice to work in a place where a developer has complete influence through the entire process. Every place I've worked has been so large that a single developer can only work on a very minute segment of the code unless they are very senior. I'm guessing the bulk of developers work at large companies and have experiences much closer to mine. This tool was probably built by several teams of developers working somewhat isolated. Someone built a system that allows for treatments to be suggested for certain ailments, another team did the UI, another testing, then there was probably completely separate team of people in charge of determining what to suggest for as treatments. This treatment team was very likely not staffed by developers at all. Which of these groups are responsible here? Do I need to be pushing back with every dialog option I create with data from a database? Ask my PO, "what are we going to do to ensure that nobody puts dangerous options in the db?" I'm being serious, if you're blaming the developers for this, then you should make suggestions that are appropriate for the real environment most of us work in; that is: we're often a very small cog in a very large machine with no view of the big picture.
idiot900 7y ago
I'm a doctor. During medical school and residency, I was taught in good faith that "only as much as necessary" opioid was OK to give and wouldn't create dependence - it was only an "excess" of opioid that would cause a problem. So under the banner of reducing the suffering of our patients, and a mandate to reduce patient-reported pain scores, opioids flowed relatively freely. Now we know that was a terrible idea with devastating consequences. But we didn't know better and truly thought we were doing right by our patients. It's hard to imagine a programmer without medical training could be blamed.
- pimmen 7y ago
  That would absolutely be a valid reason to build the service like that; the medical community thought that only an excess of opioids would be a problem, and it directly improves the numbers they're evaluated on. However, once the medical community gets wise to what an awful idea that was, I hope that when the software providers hear about this they try to figure out a way to mitigate the damage they're causing. This feature was added in 2016. Though, I think we agree that you cannot make an ethical decision on which action you should take unless you have any idea of what the consequences of your actions will be.
- bpaddock 7y ago
  The problem is things have gone to far the other way now. Those with Chronic Pain are now committing suicide (as did my wife) when they are let to suffer. The CDC itself stated in April of 2019: "CDC Advises Against Misapplication of the Guideline for Prescribing Opioids for Chronic Pain Some policies, practices attributed to the Guideline are inconsistent with its recommendations" Sadly the medical community is not getting that message due to this mater now becoming political. https://www.cdc.gov/media/releases/2019/s0424-advises-misapplication-guideline-prescribing-opioids.html https://www.cdc.gov/media/releases/2019/s0424-advises-misapp...
  - conanbatt 7y ago
    Incredible that this is another comment of yours that manifests the answer I gave you in this topic. Providers are under the microscope for opioids now, so why risk it, it's not their pain. Medicine is a rough field.

by pimmen 7y ago

nkrisc 7y ago

To be fair to the developers and designers: how exactly would they know that the inclusion of "opioid prescription" as an item in that list would help fuel a an opioid epidemic? Hindsight is 20/20, but would you expect them at the time to think, "If we include this as an option in this list, it will contribute to a nationwide epidemic of opioid abuse and countless deaths"? Honestly, I think it's a stretch, especially if they weren't aware of a deal with opioid manufacturers being the reason for it in the first place. It would be one thing of some product manager came to them and said, "since a large opioid manufacturer has paid us a lot of money, we're going to implement this feature in order to increase unnecessary opioid prescriptions" but I would be surprised if that happened. "We're going to implement this feature to help doctors pick the right treatment for a patient's pain." Someone here is evil and deserves what's coming, but I don't know that it necessarily would have been readily obvious to those implementing it. In fact, it would likely have been preferable to conceal the true nature of it lest any of them have any moral qualms or objections. I'm not trying to negate your question, either. I think it's a good question to ask and consider.

mehlmao 7y ago
This "feature" was added to the software in 2016, well into the epidemic.
ryandrake 7y ago
I’ve never been a big fan of the “I don’t know what my software does—I just move JSON around” excuse. As professionals, we need to be aware of the intended use of the things we make, and that includes weighing the ethical implications. We also need a professional ethical vow with teeth like the Hippocratic Oath that allows tech professionals to push back against demands to write unethical software, but that’s a topic for another thread.
- nkrisc 7y ago
  I don't disagree. But my point was that it's not always clear what the implications are. Sometimes it's more obvious, sometime it's not. We also don't know what the "intended" use of this feature that was presented to the people who made it was. They could very well have been deceived as to it's true nature. This could very easily have been presented and sold to a team in a way that does not make it immediately seem like something evil.
  - mcguire 7y ago
    The bottom line is that developers and designers are not professionals, who would have to accept responsibility for their products. It is in the best interests of developers and designers not to be professionals and not to accept responsibility in cases like this. A standard argument for this state of affairs is that it is in the best interest of society to allow developers and designers this freedom from responsibility, because of all the shiny new products. In which case you have to accept incidents like this---you can't make an omelette without breaking a few eggs.
jfarmer 7y ago
> If we include this as an option in this list, it will contribute to a nationwide epidemic of opioid abuse and countless deaths? Give. me. a. break! At a minimum they should ask, "What are the potential downsides? How extreme are they? How can we determine how likely those potential downsides are?" I'd expect every junior PM to ask _that_ series of questions about any product decision in any market. There's a reason clinical trials have protocols for early termination. Not understanding that early termination is a thing when building a social app is one thing, but they're building EMR software!
- mywittyname 7y ago
  Must be nice to work in a place where a developer has complete influence through the entire process. Every place I've worked has been so large that a single developer can only work on a very minute segment of the code unless they are very senior. I'm guessing the bulk of developers work at large companies and have experiences much closer to mine. This tool was probably built by several teams of developers working somewhat isolated. Someone built a system that allows for treatments to be suggested for certain ailments, another team did the UI, another testing, then there was probably completely separate team of people in charge of determining what to suggest for as treatments. This treatment team was very likely not staffed by developers at all. Which of these groups are responsible here? Do I need to be pushing back with every dialog option I create with data from a database? Ask my PO, "what are we going to do to ensure that nobody puts dangerous options in the db?" I'm being serious, if you're blaming the developers for this, then you should make suggestions that are appropriate for the real environment most of us work in; that is: we're often a very small cog in a very large machine with no view of the big picture.

idiot900 7y ago

I'm a doctor. During medical school and residency, I was taught in good faith that "only as much as necessary" opioid was OK to give and wouldn't create dependence - it was only an "excess" of opioid that would cause a problem. So under the banner of reducing the suffering of our patients, and a mandate to reduce patient-reported pain scores, opioids flowed relatively freely. Now we know that was a terrible idea with devastating consequences. But we didn't know better and truly thought we were doing right by our patients. It's hard to imagine a programmer without medical training could be blamed.

pimmen 7y ago
That would absolutely be a valid reason to build the service like that; the medical community thought that only an excess of opioids would be a problem, and it directly improves the numbers they're evaluated on. However, once the medical community gets wise to what an awful idea that was, I hope that when the software providers hear about this they try to figure out a way to mitigate the damage they're causing. This feature was added in 2016. Though, I think we agree that you cannot make an ethical decision on which action you should take unless you have any idea of what the consequences of your actions will be.
bpaddock 7y ago
The problem is things have gone to far the other way now. Those with Chronic Pain are now committing suicide (as did my wife) when they are let to suffer. The CDC itself stated in April of 2019: "CDC Advises Against Misapplication of the Guideline for Prescribing Opioids for Chronic Pain Some policies, practices attributed to the Guideline are inconsistent with its recommendations" Sadly the medical community is not getting that message due to this mater now becoming political. https://www.cdc.gov/media/releases/2019/s0424-advises-misapplication-guideline-prescribing-opioids.html https://www.cdc.gov/media/releases/2019/s0424-advises-misapp...
- conanbatt 7y ago
  Incredible that this is another comment of yours that manifests the answer I gave you in this topic. Providers are under the microscope for opioids now, so why risk it, it's not their pain. Medicine is a rough field.