5 ms·

I would say misuse AI for mental health advice. It's already difficult to find good mental health professionals.

by abirch 8d ago

fn-mote 8d ago
Those two sentences seem to be a non-sequitor. Unavailability is one reason people would turn to AI. Ease of access is another.
- simonwsimonw 8d ago
  [dead]
Cthulhu_ 8d ago
Exactly why people turn to AI, which I think is good enough for "casual" mental health advice (the kind people can do themselves by reading self-help books or talking with empathic friends). Arguably it's safer than talking to strangers online because now that they've been doing it for a few years, they will (or should) have plenty of safeguards, unlike people on the internet who can talk out of their ass (like me!) and give bad advice. In theory the LLM will escalate or recommend professional help if they're out of their depth.
- newswasboring 8d ago
  > In theory the LLM will escalate or recommend professional help if they're out of their depth. LLMs can barely identify gaps in their own software designs, and they are provably good at making software.
  - duskwuff 7d ago
    LLMs are also incredibly poor at recognizing that they're "out of their depth", especially when their context is already full of their attempts to do something.
- officeplant 8d ago
  >In theory the LLM will escalate or recommend professional help if they're out of their depth. Given there have been multiple suicides driven by LLMs. I really doubt that.
  - tt24 7d ago
    Fascinating perspective - any technology that has caused >0 deaths is a dead end and has no chance of improvement.
    - officeplant 7d ago
      I'm sure the Torment Nexus can be improved.
    - pocksuppet 6d ago
      Really hot take apparently: any technology that causes people to commit suicide during therapy isn't a good therapist.
- micromacrofoot 8d ago
  In practice we know that LLMs have aided in suicide, so what they should do in theory doesn't matter.
  - voakbasda 7d ago
    And how many people can it be credited for saving from self harm? Probably countless more than it has harmed, if I had to bet.
deleted 8d ago
[deleted]
nekusar 8d ago
I've yet to find anybody in mental health profession who is "good", aside psychiatrists and their drug prescriptions. And that's only cause of gatekeeping drugs, not because they're good either. The worst problem with mental health care is that there is no "you're better" metric. There is if your bone breaks and heal. There is for surgery of various things. But mental health seems to follow Parkinson's Law, and expands to the number of sessions available, whether it helps or not. And at X00$/session, yeah gets costly quick. So if something trained on all psychiatric literature costs $20/month subscription, that's a substitute therapist. Whether it's clinically trained or not.
- cheesecakegood 7d ago
  I’ve seen about 8 therapists and 3 coaches at various times in the last 15 years. 5-6/8 and 2/3 respectively were, I eventually realized, mostly in it for the money and didn’t actually care too much (if at all) about me making any positive changes permanent. It wasn’t all bad; having someone confidential to talk to is in itself valuable, but the cost/benefit ratio is not at all consistent. With that kind of benchmark comparison, it’s not all that hard for AI to present some kind of favorable value proposition, even if also inconsistent but with a harder ceiling on the upside.
miki123211 8d ago
Unless you have Bill Gates money, you don't have access to a mental health professional (much less your mental health professional who knows your life situation) when a girlfriend dumps you over text at 1AM. And that's when you may need one most. People want to talk to their therapist after an argument with their husband or when their boss fires them, not Friday after next.
- newswasboring 8d ago
  The need is not in question, the solution is. This annoying glaze machine is not it for mental health tasks.
  - nish__ 8d ago
    Most people have friends and family to "glaze" them anyway. I don't see it as a major danger. Also, it's gotten better with the glazing lately anyway.
    - hatsix 8d ago
      The danger is in the replacement. Confiding in others builds a web of trust. Talking about a break-up with a friend builds a relationship, and the next time that friend needs someone, they know you're an option. It's economics, borrowing and trading emotional support. AI as confidant destroys that economy.
  - p-e-w 7d ago
    Tens of millions of people disagree with you. What makes you think you know better than them what is right for them?
    - z0r 7d ago
      This line of reasoning doesn't hold up for many harmful habits that people voluntarily partake in. It's OK for someone to believe that using an AI as an approximate therapist might be a bad idea, especially in light of the many publicly documented cases of AI feeding delusional thoughts and encouraging suicide.

I would say misuse AI for mental health advice. It's already difficult to find good mental health professionals.

by abirch 8d ago

I would say misuse AI for mental health advice. It's already difficult to find good mental health professionals.

Those two sentences seem to be a non-sequitor. Unavailability is one reason people would turn to AI. Ease of access is another.

simonwsimonw 8d ago
[dead]

Cthulhu_ 8d ago

Exactly why people turn to AI, which I think is good enough for "casual" mental health advice (the kind people can do themselves by reading self-help books or talking with empathic friends). Arguably it's safer than talking to strangers online because now that they've been doing it for a few years, they will (or should) have plenty of safeguards, unlike people on the internet who can talk out of their ass (like me!) and give bad advice. In theory the LLM will escalate or recommend professional help if they're out of their depth.

newswasboring 8d ago
> In theory the LLM will escalate or recommend professional help if they're out of their depth. LLMs can barely identify gaps in their own software designs, and they are provably good at making software.
- duskwuff 7d ago
  LLMs are also incredibly poor at recognizing that they're "out of their depth", especially when their context is already full of their attempts to do something.
officeplant 8d ago
>In theory the LLM will escalate or recommend professional help if they're out of their depth. Given there have been multiple suicides driven by LLMs. I really doubt that.
- tt24 7d ago
  Fascinating perspective - any technology that has caused >0 deaths is a dead end and has no chance of improvement.
  - officeplant 7d ago
    I'm sure the Torment Nexus can be improved.
  - pocksuppet 6d ago
    Really hot take apparently: any technology that causes people to commit suicide during therapy isn't a good therapist.
micromacrofoot 8d ago
In practice we know that LLMs have aided in suicide, so what they should do in theory doesn't matter.
- voakbasda 7d ago
  And how many people can it be credited for saving from self harm? Probably countless more than it has harmed, if I had to bet.

deleted 8d ago

[deleted]

nekusar 8d ago

I've yet to find anybody in mental health profession who is "good", aside psychiatrists and their drug prescriptions. And that's only cause of gatekeeping drugs, not because they're good either. The worst problem with mental health care is that there is no "you're better" metric. There is if your bone breaks and heal. There is for surgery of various things. But mental health seems to follow Parkinson's Law, and expands to the number of sessions available, whether it helps or not. And at X00$/session, yeah gets costly quick. So if something trained on all psychiatric literature costs $20/month subscription, that's a substitute therapist. Whether it's clinically trained or not.

cheesecakegood 7d ago
I’ve seen about 8 therapists and 3 coaches at various times in the last 15 years. 5-6/8 and 2/3 respectively were, I eventually realized, mostly in it for the money and didn’t actually care too much (if at all) about me making any positive changes permanent. It wasn’t all bad; having someone confidential to talk to is in itself valuable, but the cost/benefit ratio is not at all consistent. With that kind of benchmark comparison, it’s not all that hard for AI to present some kind of favorable value proposition, even if also inconsistent but with a harder ceiling on the upside.

miki123211 8d ago

Unless you have Bill Gates money, you don't have access to a mental health professional (much less your mental health professional who knows your life situation) when a girlfriend dumps you over text at 1AM. And that's when you may need one most. People want to talk to their therapist after an argument with their husband or when their boss fires them, not Friday after next.

newswasboring 8d ago
The need is not in question, the solution is. This annoying glaze machine is not it for mental health tasks.
- nish__ 8d ago
  Most people have friends and family to "glaze" them anyway. I don't see it as a major danger. Also, it's gotten better with the glazing lately anyway.
  - hatsix 8d ago
    The danger is in the replacement. Confiding in others builds a web of trust. Talking about a break-up with a friend builds a relationship, and the next time that friend needs someone, they know you're an option. It's economics, borrowing and trading emotional support. AI as confidant destroys that economy.
- p-e-w 7d ago
  Tens of millions of people disagree with you. What makes you think you know better than them what is right for them?
  - z0r 7d ago
    This line of reasoning doesn't hold up for many harmful habits that people voluntarily partake in. It's OK for someone to believe that using an AI as an approximate therapist might be a bad idea, especially in light of the many publicly documented cases of AI feeding delusional thoughts and encouraging suicide.