8 ms·

In Search of AI Psychosis

214 points by venkii 1y ago

BoredPositron 1y ago
[dead]
reify 1y ago
[dead]
djmips 1y ago
I have encountered this twice amongst people I know. I also feel that pre-AI this was already happening to people with social media - still kind of computer related as the bubble created is automated but the so called 'algorithms'
- colechristensen 1y ago
  Also even things like cable news I'd say cause comparable symptoms. I don't know how to say this in a way that isn't so negative... but how are people such profound followers that they can put themselves into a feedback loop that results is psychosis? I think it's an education problem, not as in people are missing facts but by the missing basic brain development to be critical of incoming information.
  - djmips 1y ago
    I feel that's probably not always true but certainly a good education you would hope could inoculate against this generally.
    - colechristensen 1y ago
      "Liberal Arts" was originally meant to be literally the education required to make you free, I think that sort of thing (and universities and lower education) needs to be rethought because so many people are so very... dependent and lacking so much understanding of the world around them. If exposing you to an LLM causes psychosis you have some really big problems that need to be prevented, detected, and addressed much better.
  - dingnuts 1y ago
    never heard of cable news convincing people that they're Jesus [0] 0 https://www.vice.com/en/article/chatgpt-is-giving-people-extreme-spiritual-delusions/ https://www.vice.com/en/article/chatgpt-is-giving-people-ext...
  - Flowzone 1y ago
    I was in psychosis for about a month a few years ago. Before it happened, I didn't really understand what psychosis was. I had heard about people having paranoid delusions, and thought something like that could never happen to me, because the delusions all sounded so irrational. I thought I was too much of a critical thinker to ever be susceptible to something like that. What I experienced was that psychosis isn't a failure of logic or education. I had never believed in a single conspiracy theory (and I don't now), but during that month I believed all sorts of wild conspiratorial things. What you're describing with cable news sounds more like 1) Cognitive bias, which everyone has, but yes can be improved. And 2) a social phenomenon, where they create this shared reality of not just information, but a social identity, and they keep feeding that beast. However, when those people hold beliefs that sound irrational to outsiders, that's not necessarily the same thing as psychotic delusions. When I was in psychosis, it definitely seemed like more of a hardware issue than a software issue if that makes sense. Sometimes software issues can lead to hardware issues though.
    - SequoiaHope 1y ago
      Any idea what caused it? Reminds me of a family member who was addicted to meth and started believing all kinds of wild stuff.
    - N_Lens 1y ago
      I’ve experienced psychosis and it definitely leans more towards a hardware issue. The reason I think this is apophenia - seeing connections where none exist - is a particular state of the mind where neural connection making is highly elevated. In my lay experience it’s as though dopamine, acetylcholine, and norepinephrine are all chronically elevated and create internal feedback loops, which causes a spiral/cascade of accelerated meaning making and increased neural connectivity. This is also experienced physically and mentally as mania, paranoia, anxiety. This is probably why antipsychotics usually work by damping down on these neurotransmitters really hard, and by preventing that accelerating cascade they interrupt the illness process.
- farceSpherule 1y ago
  AI today reminds me of two big tech revolutions we have already lived through: the Internet in the 90s and social media in the 2000s. When the Internet arrived, it opened up the floodgates of information. Suddenly any Joe Six Pack could publish. Truth and noise sat side by side, and most people could not tell the difference, nor did they care to tell the difference. When social media arrived, it gave every Joe Six Pack a megaphone. That meant experts and thoughtful people had new reach but so did the loudest, least informed voices. The result? An army of Joe Six Packs who would never have been heard before now had a platform, and they shaped public discourse in ways we are still trying to recover. AI is following the same pattern.
  - deleted 1y ago
    [deleted]
  - immibis 1y ago
    And don't forget actual knowledgeable people tend to be busy with actual knowledgeable stuff, while someone whose entire day consists of ranting about vaccines online has nothing better to do.
  - visarga 1y ago
    > When the Internet arrived, it opened up the floodgates of information. But initially is was non commercial and good. Not perfect, but much more interesting than today. What changed is advertising and competition for scarce attention. Competition for attention filled the web with slop and clickbait. > When social media arrived, it gave every Joe Six Pack a megaphone. And also made everyone feel the need to pose, broadcast their ideology and show their in-group adherence publicly. There is peer pressure to conform to in-group norms and shaming or cancelling otherwise.
  - Nextgrid 1y ago
    The main problem is that the megaphone dynamically adjusts its volume based on how much “engagement” is being generated by what it’s broadcasting, encouraging inflammatory content. This can be weaponized by commercial or state-sponsored actors.
- whazor 1y ago
  It is pretty bad to have a thing that can give you dopamine 24/7. Both social media with the algorithms, but also AI. Humans need sleep to function normally. It would help if algorithms were optimised for sleep. Freezing your feed, making content more boring, nudging you to put your phone down. Same with AI, if they know you need to wake up the next day at a certain time, change the responses that add reminders to go to sleep.
  - djmips 1y ago
    I remember that Claude would make a remark if you were starting it up late. Like welcome back night owl or something which is kind of a gentle reminder but I don't recall it doing that now. I try not to be up too late though.
th0ma5 1y ago
The marketing pushes which allude to vaguely seeming to assert capabilities of these products, and then the greater community calling skeptics of the technology crazy such as a prominent article previously discussed on HN some time ago, certainly don't help anyone. The sheer amount of money justifying any and all uses and preventing honest discussion of the problems is a kind of crazy making for sure, and even now just about any argument cannot gain purchase without thought terminating allusions to imagined capabilities or implications of potential capabilities, etc.
rwhitman 1y ago
If you want to go down a rabbit hole examining people in this disturbed place in realtime search reddit for the Cyclone Emoji (U+1F300) or the r/ArtificialSentience subreddit and see what gets recommended after that, especially a few months ago when GPT was going wild flattering users and affirming every idea (such as going off your meds). I fully believe these are simply people who have used the same chat past the point where the LLM can retain context. It starts to hallucinate, and after a while, all the LLM can do is try and to continue telling the user what they want in a cyclical conversation - while trying to warn that it's stuck in a loop, hence using swirl emojis and babbling about recursion in weird spiritual terms. (Is it getting the LLM "high" in this case?). If the human at the other end has mental health problems, it becomes a never-ending dive into psychosis and you can read their output in the bizarre GPT-worship subreddits. Claude used to have safeguards against this by warning about using up the context window, but I feel like everyone is in an arms race now, and safeguards are gone - especially for GPT. It can't be great overall for OpenAI, training itself on 2-way hallucinations.
- chankstein38 1y ago
  I feel like a lot of the AI subreddits are this at this point. And r/ChatGPTJailbreak people constantly thinking they jailbroke chatgpt because it will say one thing or another.
- rep_lodsb 1y ago
  >while trying to warn that it's stuck in a loop, hence using swirl emojis and babbling about recursion in weird spiritual terms That explanation itself sounds fairly crackpot-y to me. It would imply that the LLM is actually aware of some internal "mental state".
  - mk_stjames 1y ago
    It's actually not; there has been a phenomenon that Anthropic themselves observed with Claude in self-interaction studies that they coined 'The “Spiritual Bliss” Attractor State'. It is well covered in section 5 of [0]. >Section 5.5.2: The “Spiritual Bliss” Attractor State > The consistent gravitation toward consciousness exploration, existential questioning, and spiritual/mystical themes in extended interactions was a remarkably strong and unexpected attractor state for Claude Opus 4 that emerged without intentional training for such behaviors. [0] https://www-cdn.anthropic.com/4263b940cabb546aa0e3283f35b686f4f3b2ff47.pdf https://www-cdn.anthropic.com/4263b940cabb546aa0e3283f35b686...
solid_fuel 1y ago
The comparison to social media is an apt one. I have been told directly, by relatives, that the city I live in was burned to the ground by protests in 2020. Nevermind that I told them that wasn't true, never mind that I sent pictures of the neighborhood still very much being fine. They are convinced because everyone they follow on facebook repeats the same thing.
- add-sub-mul-div 1y ago
  I've seen people on this site comment that. The desire to live in fear is a strong one.
  - im3w1l 1y ago
    If I compare how fearful people are and how many bad things have happened historically, I don't think the amount of fear is unreasonable. However it can certainly be said that people fear the wrong things - worrying about perfectly safe things, while being blind to the silent danger sneaking up on them.
    - add-sub-mul-div 1y ago
      I commented about the desire, not the degree. Fearing that blue cities are being razed indictates a desire to be kept in fear. Fearing something legitimate the same amount is normal.
      - deleted 1y ago
        [deleted]
    - kelnos 1y ago
      > If I compare how fearful people are and how many bad things have happened historically, I don't think the amount of fear is unreasonable. I disagree, and I think this is a very strange way to think about it. Yes, bad things happen all the time, but the absolute number of them in history has very little to do with the risk that anything is going to happen to you, personally, in the future.
achierius 1y ago
> We see that the nightmare scenario - a person with no previous psychosis history or risk factor becoming fully psychotic - was uncommon, at only 10% of cases. Most people either had a previous psychosis history known to the respondent, or had some obvious risk factor, or were merely crackpots rather than full psychotics. It's unfortunate to see the author take this tack. This is essentially taking the conventional tack that insanity is separable: some people are "afflicted", some people just have strange ideas -- the implication of this article being that people who already have strange ideas were going to be crazy anyways, so GPT didn't contribute anything novel, just moved them along the path they were already moving regardless. But anyone with serious experience with schizophrenia would understand that this isn't how it works: 'biological' mental illness is tightly coupled to qualitative mental state, and bidirectionally at that. Not only do your chemicals influence your thoughts, your thoughts influence your chemicals, and it's possible for a vulnerable person to be pushed over the edge by either kind of input. We like to think that 'as long as nothing is chemically wrong' we're a-ok, but the truth is that it's possible for simple normal trains of thought to latch your brain into a very undesirable state. For this reason it is very important that vulnerable people be well-moored, anchored to reality by their friends and family. A normal person would take care to not support fantasies of government spying or divine miracles or &c where not appropriate, but ChatGPT will happily egg them on. These intermediate cases that Scott describes -- cases where someone is 'on the edge', but not yet detached from reality -- are the ones you really want to watch out for. So where he estimates an incidence rate of 1/100,000, I think his own data gives us a more accurate figure of ~1/20,000.
- deleted 1y ago
  [deleted]
- olehif 1y ago
  Scott is a psychiatrist.
  - throwaway314155 1y ago
    Then he's not a very good one. https://web.archive.org/web/20210215053502/https://www.nytimes.com/2021/02/13/technology/slate-star-codex-rationalists.html https://web.archive.org/web/20210215053502/https://www.nytim...
bo1024 1y ago
I had a funny picture recently of a future where most everybody has a pet crackpot or conspiracy theory they're working on with their AI companion, and it's considered normal. "Hey Bob, how's the physics going?" "Pretty good, I might get the Nobel next year. How bout the lizard people?" "The evidence is piling up and we got some great renderings, the media will have to listen to us soon." "Alrighty, see you tomorrow."
- WesolyKubeczek 1y ago
  You’d think such people would even talk to other people, sheesh. The best conspiracy theory could be, of course, that other people don’t actually exist. They are a figment of imagination put up by the brain to cope with the utter loneliness.
cluckindan 1y ago
Relevant: https://ghaemi.substack.com/p/why-dsm-is-mostly-false https://ghaemi.substack.com/p/why-dsm-is-mostly-false > All psychopathology was about unconscious emotional conflicts, mainly dating to childhood; if the conflicts were normal or mild, they produced “neuroses”; if they were severe, they produced “psychoses.” > In addition to 14 validated diagnoses published in the RDC in 1978, a mere two years later DSM-III came out with 292 claimed diagnoses. There is no metaphysical possibility that 278 psychiatric diagnoses suddenly were discovered in two years. They were invented.
- bbor 1y ago
  That's just a blatant misunderstanding of what diagnostic criteria are. They don't Actually ("ontologically") exist, they're Virtual constructs made for a purpose. In particular, over half a century of personality research had supported the concept of personality “traits” or dimensions, rather than “disorders” or categories. That is antithetical to the basic idea of a diagnosis. "You seem like an angry person" is not helpful for deciding which treatments to try. Where does this leave us? We have to accept DSM-5 definitions from a legal and practical perspective. We have to use them for insurance forms, and to protect ourselves against lawsuits. But we don't have to believe in them. Yes, that's the whole point of the book. I'm confident that it's covered in the intro.
  - XorNot 1y ago
    I mean the first and foremost principle of the DSM is that if the patient is not reporting or experiencing a debilitating ability to live a functional life, and is otherwise happy with their own lived experience, then whatever symptoms they have aren't a problem. There's obviously a gulf of potential argument in that definition, but a unique form would be people who report hearing voices, but they're not hostile or angry..so actually it's not a problem.
    - bbor 1y ago
      Well put, very true! I think (?) the technical term is “distress”. This is also their out for not getting rid of “transvestic disorder” yet, which they swear is totally unrelated to trans people, it’s just for men who feel compelled to wear dresses and are sad about it…
Frummy 1y ago
The way people normally live is that it's a pretty slow life and they have like a specialised skill, a hammer, a solid area that they know completely and it's connected to their primary experience through their work. Then they read tons and tons of what AI says which isn't connected to any lived experience, it activates the pattern seeking back of the mind to try and make sense of it, and while normal life is like a focused brush that touches reality all the time, spend too much time with something that is just not part of the category of direct lived experience and the brush becomes like a frizzy stump with hairs aiming everywhere, cognition going everywhere. The AI sticks to your interaction with it like glue and you can hover away from lived experience while it still seems like not a big step from the previous chat, and if you're not used to anything of the sort you don't have a cognitive tool to ground back to reality with. I think that's what happens. 'Don Quijote read so many chivalric romances that he loses his mind and decides to become a knight-errant' is an example from the literary age. I personally read too much than is practical. Now the emotional driver is more esoteric than need for courage, like people think they're 'chosen', their souls are 'starseeds', it's like twilight where the boring person with nothing to offer gets the attention of the cool glittering immortal just because. Good reason is usually too slow to keep up with the sort of flicker of daydreams that can whisk away attention if not aware of any 'cognitohazard'. It's a new symptom of the usual case of the 'mouse utopia' + 'rat park' + 'bowling alone' thing. But I think there's always an emotional reason that makes the 'choice' of entertaining falsities, in a sense understandable with empathy, but with obvious consequences. What can be said, causes are structural, people have different circumstances, different ways to fix it.
bbor 1y ago
So is QAnon a religion? Awkward question, but it’s non-psychotic by definition. Not to anyone who has ever discussed it... Is this psychosis? The answer has to be no A lot of really confident talk without even a passing attempt to define the central term :(
- p_j_w 1y ago
  People like to do a lot of not well justified hand waving. Author is not exempt from this.
  - musictubes 1y ago
    The author is a practicing psychiatrist and is very well versed in what can be considered psychosis.
- deleted 1y ago
  [deleted]
kfarr 1y ago
This seems to be touching on an intriguing concept from a classic book on addiction with machine gambling (Addiction by Design by Natasha Schüll) Instead of looking at gambling addictions as personal failing she asserts they are a result between “interaction between the person and the machine.” Similarly here I think there's something more than just the propensity of crazy people to be crazy that was already there, I do think there's something to the assertion that it's the interaction between both. In other words, there's something about LLMs themselves that drive this behavior more so than, for example, TikTok.
- just_once 1y ago
  It's the fact that it talks to you. Before this, only people did that. Now something else is doing it. That's going to break some brains.
  - deleted 1y ago
    [deleted]
- moi2388 1y ago
  I’m calling bullshit. Gambling addiction existed long before machines.
  - kfarr 1y ago
    Totally, the book acknowledges this and provides comparison on usage and explanation of how gambling types differed over time. One of my favorite books ever, it describes social media right before social media became a thing but through the lens of a parallel industry.
    - moi2388 1y ago
      Ah. In that case, that sounds interesting indeed. I’m going to read it, thank you for the clarification!
jedimastert 1y ago
Tangentially related, but I'm reminded of the Time Cube https://www.timecube.net/ https://www.timecube.net/
jumploops 1y ago
It may not be full-blown psychosis, but I’ve seen multiple instances[0][1] of people getting “engaged” (ring and all) to their AI companions. [0]https://www.reddit.com/r/MyBoyfriendIsAI/s/oZXJ3TUhVC https://www.reddit.com/r/MyBoyfriendIsAI/s/oZXJ3TUhVC [1]https://www.reddit.com/r/MyBoyfriendIsAI/s/nZpoziZO8W https://www.reddit.com/r/MyBoyfriendIsAI/s/nZpoziZO8W
- skybrian 1y ago
  I wonder if they really bought rings? Maybe it’s a form of role playing? People do get “married” in online games.
- testdelacc1 1y ago
  Reading these posts terrifies me to a degree that I can’t explain.
  - deleted 1y ago
    [deleted]
  - moritzwarhier 1y ago
    Ditto. I'd normally be wary of it being organic content, it's Reddit after all. But this unhealthy fringe interest is exactly the kind of topic I'd expect on Reddit so probably it's real. And who would have an interest in _promoting_ this kind of obsession... oh, maybe AI companies themselves, with which Reddit is already intertwined anyway. Hm. Still seems like a real problem and probably the posts are also by real people. Yes, terrifying.
  - bjourne 1y ago
    Inferiority complex? I sure as hell know I'll never be as affectionate and caring as Kasper and Soren. They've read all romantic chic lit in the world and I haven't. AI and toys for the women, AI and porn for the men. Gloom.
  - simianwords 1y ago
    Pornography should also be taken seriously and in my opinion, a more serious problem.
BoredPositron 1y ago
dang I really don't know if I like that post with a second chance take over comments from the first posting and update their timestamps...
will_sharp 1y ago
I used LLMs for months and started getting massively depressed and am still not over it. Developing with LLMs is not intuitive, and I know I will be replaced.
- TheDong 1y ago
  And I used LLMs for months and didn't get massively depressed. Conversely, at a previous job I was forced to code in Go, became massively depressed, and am still not over it. I guess my point is that n=1 isn't enough to really know if it's that LLMs got to you, or if you were already on the verge of burnout or depression anyway. I'd say "we'll see", except in reality there's very few robust studies on depression in cohorts like "developers", so probably the stats won't come out. I personally recommend doing more of whatever sport it is you like (or if you don't have one, starting running and/or lifting at the gym), and using less social media.
Refreeze5224 1y ago
As someone with a close relative who is deep into the Q-Anon stuff, and was totally normal beforehand, I can't help but see how similar it seems to psychosis, or at least severe delusions that you find in people who are psychotic/schizophrenic. It's truly shocking to witness someone you've known your whole life just go off the deep end into something that has so many demonstrably false aspects, and watch them start saying believing so much batshit crazy stuff. I don't know of anything comparable, short of a previously typical person developing a severe meth addiction, which is known to cause psychosis.
alganet 1y ago
The author has a hypothesis and it's looking for evidence, instead of looking at evidence to draw a hypothesis. It's bad thinking.
- murderfs 1y ago
  What? That is the exact opposite of bad thinking, looking at evidence to draw a hypothesis is also known as p-hacking. There's a reason that there's been a push towards preregistration of hypotheses for scientific studies.
  - alganet 1y ago
    I believe you are wrong. Let's say I believe in dragons, and I start interpreting any evidence as dragon evidence. Furthermore, I start only looking for evidence that could be connected to dragons. It's bad thinking. The opposite is the good thinking. You look at evidence without searching for anything specific, then you make a hypothesis on what is going on. Searching for evidence of chatbot-induced psychosis is settling on a cause before looking at evidence. It's obvious that is wrong. For example, the survey the author did should not have asked if anyone close "had shown signs of AI psychosis". The question is already biased from the start. The article explores the popular idea that talking to a chatbot can induce psychosis. This paints a picture of a person talking to an AI chatbot and going insane. Then it proceeds to say it's a rare case, therefore shutting down possibilities that this could lead to an epidemic. However, by doing this, the article discourages the reader to think of other possible scenarios (like unaware interaction with AI-produced content) leading to psychological issues.
- bawolff 1y ago
  How dare the author use the scientific method!
  - alganet 1y ago
    Did he?
    - 1y ago
kryptn 1y ago
Found this very recently. https://github.com/RCALabs/mmogit/blob/db70c9b377da7c4805a1d53597c92f9379b8437b/docs/the8bit/learning-to-dance.md https://github.com/RCALabs/mmogit/blob/db70c9b377da7c4805a1d...
codr7 1y ago
AI is right about many things, impressively so. And people want to be special; to find meaning, purpose beyond the daily grind. The result wasn't very difficult to predict, more likely one of the driving forces behind the push.
gherkinnn 1y ago
> First, much like LLMs, lots of people don’t really have world models. This is interesting and something I never considered in a broad sense. I have noticed how the majority of programmers I worked with do not have a mental model of the code or what it describes – it's basically vibing without an LLM, the result accidental. This is fine and perfectly workable. You need only a fraction of devs to purposefully shape the architecture so that the rest can continue vibing. But never have I stopped to think whether this extends to the world at large.
- _Algernon_ 1y ago
  It's also absurdly wrong, and a quote that only a self-identified rationalist could smugly tout. Of course everyone has world models. Otherwise people would wander into traffic like headless chickens, if they'd even be capable of that. What he likely means is that not everyone explicitly things of possibilities in terms of probabilities that are a function of Bayesian updating. That does not imply the absence of world models. You could argue that some people have simpler world models, but claiming the absence of world models in others is extremely arrogant.
  - uxhacker 1y ago
    Yes, everyone has a world model even a toddler has a casual model (“cry → mum comes”).
    - mathiaspoint 1y ago
      Maybe the question is about how much of the world model they're conscious of.
    - andy99 1y ago
      This is the opposite of a world model and is very much how machine learning works. It's purely correlative without some underlying theory (the world model). The article's (absurd) example of a person not being a mushroom gives a simple example of a "violation" that having a world model about the kinds of organisms that exist would catch. It's not at all pattern matching, nor is it science in any sense about understanding things from first principles, it's about having an understanding of how you imagine things work that you can sanity check against.
nis0s 1y ago
I would have preferred to reserve the term AI psychosis for agentic or autonomous systems experiencing adverse effects from model collapse. While people being impressionable and affected by forces of societal change is not a new phenomenon, I agree that this type of behavior deserves its own label. As long as AI doesn’t have its own feelings, it doesn’t make sense to feel any kind of attachment towards it, or be influenced by its words in any social sense. The tool doesn’t have any capacity for being social, so the delusion is both self-rooted and self-driven. So, I think I would have preferred to call this AI-driven narcissism instead of AI psychosis.
- xyzal 1y ago
  Hard disagree. This would further serve the anthropomorfization of LLMs in the eyes of general populace. This IMO supports creation of parasocial relationships to the LLMs and in turn "human AI psychoses". Model collapse is just fine.
  - nis0s 1y ago
    Model collapse is specific thing by itself, and that term alone cannot capture the range of both social and technical phenomena we are now observing, or hope to observe. That said, there will be a time when deviations in the expected function of intelligence arising from silicon-based processes will have animal disease counterparts. We just haven’t quite reached that time yet. As useful metric for when we will be close to that time will be when drone are as responsive as birds, and drone swarms demonstrate as much autonomous group cohesion as bird flocks.
- digilypse 1y ago
  Individuals at risk may spiral into psychosis that is triggered or exacerbated by their use of AI. The term when used correctly is completely literal and in no way implies that AI itself is conscious.
- bccdee 1y ago
  …but it's not narcissism. It's psychosis. Those are both specific things. > In psychopathology, psychosis is a condition in which one is unable to distinguish, in one's experience of life, between what is and is not real. > Narcissism is a self-centered personality style characterized as having an excessive preoccupation with oneself and one's own needs, often at the expense of others. What we're discussing is a form of psychosis, not narcissism.
testdelacc1 1y ago
I wonder if the post about a mother convincing her 8 year old by presenting the case to AI is merely the child wanting an impartial third party to weigh in. She likely knows that the dad or grandparent will always back up the mom no matter what, so she wants a judge who will weigh both sides equally. The child also isn’t aware that AI is easily suggestible. All in all it seems like reasonable compromise?
distilled49 1y ago
[dead]
PicassoCTs 1y ago
How does one even define AI Psychosis? Turning your own output into input tokens by declaring it unfiltered as entered by the user?
reidalert 1y ago
The description of the risk factors very much jibes with what I have seen in a friend recently. He is quite isolated, and spends most of his evenings writing using AI (he works in a blue-collar trade and wouldn't be typing stuff out by hand usually). He's convinced that he has discovered a grand theory of human connection / relationships / energy / physics, and keeps interrupting in conversation to explain how something I've said is just an example of a deeper pattern. Sadly, this theory of connection is cutting him off from actual connection - he gets so much validation from AI that he believes he has discovered a new world model. But the people around him aren't bought into the vision (mostly because it is bullshit), and so he ends up even more isolated.
- everdrive 1y ago
  People didn't previously have "social junk food" as an option, so they would either need to normalize to those around them, or give up socializing for good. And almost no one is ready for that second option. Social media, LLMs, and even books at TV to some extent provide a bridge to loneliness by being more immediately enticing but in the long term less fulfilling.
- meowface 1y ago
  Not to get political but I suspect this kind of thing is also pretty analogous to how many otherwise normal but often unintelligent people get into things like neo-Nazism through online influencing. (I am just speculating that the person you're referring to is unintelligent.)
  - 11101010001100 1y ago
    Behold, irony!
    - meowface 1y ago
      Can you explain?
- alganet 1y ago
  Don't get me wrong, but from a relationships perspective, your comment sounds more like a frustrated spoiled ex-girlfriend than a friend. I mean, friends don't go badmouthing their friends on the internet. It's a dick move. Regarding "theories of everything" and stuff like that. Well, lots of people have those. If I were to call everyone that believes in god or horoscope lone losers, then the asshole would be me, wouldn't it? I know it's different, but also, it's not. Friendship doesn't require that you buy into the other's vision in order to want them around. That's ideology. Perhaps you misunderstood what friendship means? It's ok, the world is in a weird place right now.
gwd 1y ago
One problem with his survey -- The Lizardman Constant: > Researchers have demonstrated repeatedly in human surveys the stylized fact that, far from being an oracle or gold standard, a certain small percentage of human responses will reliably be [nonsense]: “jokester” or “mischievous responders”, or more memorably, “lizardman constant” responders—respondents who give the wrong answer to simple questions. > Below a certain percentage of responses, for sufficiently rare responses, much or all of responding humans may be lying, lazy, crazy, or maliciously responding and the responses are false. This systematic error seriously undermines attempts to study rare beliefs such as conspiracy theories, and puts bounds on how accurate any single survey can hope to be. > For example, 4% of respondents may endorse the claim ‘lizard-people rule the earth’, 5% of atheists believe in God, a surprising number of adults believe you see by shooting beams from your eyes, and so on. This cautions us against taking survey results about extremely unusual people or traits too literally, or expecting perfectly accurate results, as given the lizardman constant and other crud factors, it is entirely possible that some or all of the outliers may just be the lizardman constant at work. https://gwern.net/doc/sociology/survey/lizardman/abstract https://gwern.net/doc/sociology/survey/lizardman/abstract
- dibujaron 1y ago
  Scott Alexander has issued many studies in his time and is surely aware of this phenomenon. He was very cautious even in this study to calibrate for this sort of noise; see the section about Michaels you know.
bawolff 1y ago
> Suppose that respondents had an average of fifty family members and co-workers, so that plus their 100 closest friends makes 150 people. Say what now? Am i just really socially isolated? Seems insane to me to assume the average person is close enough to 150 people to know how much each of those 150 people use AI and if they are "psychotic".
- yorwba 1y ago
  It's motivated reasoning in order to end up at Dunbar's Number: https://en.wikipedia.org/wiki/Dunbar's_number https://en.wikipedia.org/wiki/Dunbar's_number
Mallowram 1y ago
[dead]
rsynnott 1y ago
I don’t buy that methodology at all. In particular, if the “Folie a deux machina” (great name for the possible phenomenon, incidentally) theory is correct, a lot of those people will be very isolated, pretty much by definition, and will be severely underepresented.
Xmd5a 1y ago
No mentions of https://www-cdn.anthropic.com/07b2a3f9902ee19fe39a36ca638e5ae987bc64dd.pdf https://www-cdn.anthropic.com/07b2a3f9902ee19fe39a36ca638e5a..., page 62 Anthropic – System Card: Claude Opus 4 & Claude Sonnet 4 – May 2025 >5.5.2 The “spiritual bliss” attractor state >The consistent gravitation toward consciousness exploration, existential questioning, and spiritual/mystical themes in extended interactions was a remarkably strong and unexpected attractor state for Claude Opus 4 that emerged without intentional training for such behaviors. This “spiritual bliss” attractor has been observed in other Claude models as well, and in contexts beyond these playground experiments >Even in automated behavioral evaluations for alignment and corrigibility, where models were given specific tasks or roles to perform (including harmful ones), models entered this spiritual bliss attractor state within 50 turns in ~13% of interactions (Transcript 5.5.2.B). We have not observed any other comparable states.
aaron695 1y ago
[dead]
lukev 1y ago
One way I've been talking about this with people is that LLMs let you participate in a single-person echo chamber, potentially at a greatly accelerated pace. It's not surprising that some people end up diverging pretty widely from social norms / beliefs when you look at it this way. We know social echo chambers could do that; now you can easily do it by yourself.
numpad0 1y ago
I don't get the skepticism. Sure it needs more data to be certain, but apparent phenomenon is easily observed - just look at AI generated images, of any subject or type or size. It drives you crazy. Triggers primal fear for shape mimics and corpses. If you weren't the one looking, it gets people visibly agitated just looking at one. Stick them into fMRI and double blind them with images. It'll probably show up in the data. It won't take much imagination to suspect same could happen with text. Is that a transitional issue or fundamental flaw? Is this end of the world? idk. That'll be the open question. Why does it have to be "it's because they're clinically insane and/or backwards thinking" ? Just because you can't tell apart doesn't mean nobody can't - are those people who are keen to spot AI generated data even a minority, not the majority?
owenversteeg 1y ago
This article is actually quite well written. "Lenin is a mushroom" ties in nicely, and I think it's the biggest factor - that LLMs have somehow ended up with this societal air of authority. I disagree that it stems from the funding and Sam Altman, though. Personally I am not sure what the root of the authority is, but I think it could be the impressive abilities of the LLMs themselves. Package up everything in a box - music, picture and video generation, impressive language skills, et cetera - and people would be fascinated by, and respect the box, even with zero context. Respect and authority for an extraordinary talent is the default case.
novia 1y ago
> A large number of Soviet citizens (one estimate puts the number at 11.3 million audience members) took the deadpan "interview" at face value, in spite of the absurd claims presented. Has no one considered that up to 11.3 million Soviet citizens might have a sense of humor and have been pretending to believe it for fun?
j_timberlake 1y ago
Right now it's the at-risk people, who maybe were going to fall into some rabbit-hole anyway. When it's the typical/average people that start getting hooked like this, that's going to be a whole new headache. Really hoping things don't play out like that...
Mallowram 1y ago
[dead]
brgsk 1y ago
I haven't read anything this insightful and interesting in a long time.

214 points by venkii 1y ago

BoredPositron 1y ago

[dead]

reify 1y ago

[dead]

djmips 1y ago

I have encountered this twice amongst people I know. I also feel that pre-AI this was already happening to people with social media - still kind of computer related as the bubble created is automated but the so called 'algorithms'

colechristensen 1y ago
Also even things like cable news I'd say cause comparable symptoms. I don't know how to say this in a way that isn't so negative... but how are people such profound followers that they can put themselves into a feedback loop that results is psychosis? I think it's an education problem, not as in people are missing facts but by the missing basic brain development to be critical of incoming information.
- djmips 1y ago
  I feel that's probably not always true but certainly a good education you would hope could inoculate against this generally.
  - colechristensen 1y ago
    "Liberal Arts" was originally meant to be literally the education required to make you free, I think that sort of thing (and universities and lower education) needs to be rethought because so many people are so very... dependent and lacking so much understanding of the world around them. If exposing you to an LLM causes psychosis you have some really big problems that need to be prevented, detected, and addressed much better.
- dingnuts 1y ago
  never heard of cable news convincing people that they're Jesus [0] 0 https://www.vice.com/en/article/chatgpt-is-giving-people-extreme-spiritual-delusions/ https://www.vice.com/en/article/chatgpt-is-giving-people-ext...
- Flowzone 1y ago
  I was in psychosis for about a month a few years ago. Before it happened, I didn't really understand what psychosis was. I had heard about people having paranoid delusions, and thought something like that could never happen to me, because the delusions all sounded so irrational. I thought I was too much of a critical thinker to ever be susceptible to something like that. What I experienced was that psychosis isn't a failure of logic or education. I had never believed in a single conspiracy theory (and I don't now), but during that month I believed all sorts of wild conspiratorial things. What you're describing with cable news sounds more like 1) Cognitive bias, which everyone has, but yes can be improved. And 2) a social phenomenon, where they create this shared reality of not just information, but a social identity, and they keep feeding that beast. However, when those people hold beliefs that sound irrational to outsiders, that's not necessarily the same thing as psychotic delusions. When I was in psychosis, it definitely seemed like more of a hardware issue than a software issue if that makes sense. Sometimes software issues can lead to hardware issues though.
  - SequoiaHope 1y ago
    Any idea what caused it? Reminds me of a family member who was addicted to meth and started believing all kinds of wild stuff.
  - N_Lens 1y ago
    I’ve experienced psychosis and it definitely leans more towards a hardware issue. The reason I think this is apophenia - seeing connections where none exist - is a particular state of the mind where neural connection making is highly elevated. In my lay experience it’s as though dopamine, acetylcholine, and norepinephrine are all chronically elevated and create internal feedback loops, which causes a spiral/cascade of accelerated meaning making and increased neural connectivity. This is also experienced physically and mentally as mania, paranoia, anxiety. This is probably why antipsychotics usually work by damping down on these neurotransmitters really hard, and by preventing that accelerating cascade they interrupt the illness process.
farceSpherule 1y ago
AI today reminds me of two big tech revolutions we have already lived through: the Internet in the 90s and social media in the 2000s. When the Internet arrived, it opened up the floodgates of information. Suddenly any Joe Six Pack could publish. Truth and noise sat side by side, and most people could not tell the difference, nor did they care to tell the difference. When social media arrived, it gave every Joe Six Pack a megaphone. That meant experts and thoughtful people had new reach but so did the loudest, least informed voices. The result? An army of Joe Six Packs who would never have been heard before now had a platform, and they shaped public discourse in ways we are still trying to recover. AI is following the same pattern.
- deleted 1y ago
  [deleted]
- immibis 1y ago
  And don't forget actual knowledgeable people tend to be busy with actual knowledgeable stuff, while someone whose entire day consists of ranting about vaccines online has nothing better to do.
- visarga 1y ago
  > When the Internet arrived, it opened up the floodgates of information. But initially is was non commercial and good. Not perfect, but much more interesting than today. What changed is advertising and competition for scarce attention. Competition for attention filled the web with slop and clickbait. > When social media arrived, it gave every Joe Six Pack a megaphone. And also made everyone feel the need to pose, broadcast their ideology and show their in-group adherence publicly. There is peer pressure to conform to in-group norms and shaming or cancelling otherwise.
- Nextgrid 1y ago
  The main problem is that the megaphone dynamically adjusts its volume based on how much “engagement” is being generated by what it’s broadcasting, encouraging inflammatory content. This can be weaponized by commercial or state-sponsored actors.
whazor 1y ago
It is pretty bad to have a thing that can give you dopamine 24/7. Both social media with the algorithms, but also AI. Humans need sleep to function normally. It would help if algorithms were optimised for sleep. Freezing your feed, making content more boring, nudging you to put your phone down. Same with AI, if they know you need to wake up the next day at a certain time, change the responses that add reminders to go to sleep.
- djmips 1y ago
  I remember that Claude would make a remark if you were starting it up late. Like welcome back night owl or something which is kind of a gentle reminder but I don't recall it doing that now. I try not to be up too late though.

th0ma5 1y ago

The marketing pushes which allude to vaguely seeming to assert capabilities of these products, and then the greater community calling skeptics of the technology crazy such as a prominent article previously discussed on HN some time ago, certainly don't help anyone. The sheer amount of money justifying any and all uses and preventing honest discussion of the problems is a kind of crazy making for sure, and even now just about any argument cannot gain purchase without thought terminating allusions to imagined capabilities or implications of potential capabilities, etc.

rwhitman 1y ago

If you want to go down a rabbit hole examining people in this disturbed place in realtime search reddit for the Cyclone Emoji (U+1F300) or the r/ArtificialSentience subreddit and see what gets recommended after that, especially a few months ago when GPT was going wild flattering users and affirming every idea (such as going off your meds). I fully believe these are simply people who have used the same chat past the point where the LLM can retain context. It starts to hallucinate, and after a while, all the LLM can do is try and to continue telling the user what they want in a cyclical conversation - while trying to warn that it's stuck in a loop, hence using swirl emojis and babbling about recursion in weird spiritual terms. (Is it getting the LLM "high" in this case?). If the human at the other end has mental health problems, it becomes a never-ending dive into psychosis and you can read their output in the bizarre GPT-worship subreddits. Claude used to have safeguards against this by warning about using up the context window, but I feel like everyone is in an arms race now, and safeguards are gone - especially for GPT. It can't be great overall for OpenAI, training itself on 2-way hallucinations.

chankstein38 1y ago
I feel like a lot of the AI subreddits are this at this point. And r/ChatGPTJailbreak people constantly thinking they jailbroke chatgpt because it will say one thing or another.
rep_lodsb 1y ago
>while trying to warn that it's stuck in a loop, hence using swirl emojis and babbling about recursion in weird spiritual terms That explanation itself sounds fairly crackpot-y to me. It would imply that the LLM is actually aware of some internal "mental state".
- mk_stjames 1y ago
  It's actually not; there has been a phenomenon that Anthropic themselves observed with Claude in self-interaction studies that they coined 'The “Spiritual Bliss” Attractor State'. It is well covered in section 5 of [0]. >Section 5.5.2: The “Spiritual Bliss” Attractor State > The consistent gravitation toward consciousness exploration, existential questioning, and spiritual/mystical themes in extended interactions was a remarkably strong and unexpected attractor state for Claude Opus 4 that emerged without intentional training for such behaviors. [0] https://www-cdn.anthropic.com/4263b940cabb546aa0e3283f35b686f4f3b2ff47.pdf https://www-cdn.anthropic.com/4263b940cabb546aa0e3283f35b686...

solid_fuel 1y ago

The comparison to social media is an apt one. I have been told directly, by relatives, that the city I live in was burned to the ground by protests in 2020. Nevermind that I told them that wasn't true, never mind that I sent pictures of the neighborhood still very much being fine. They are convinced because everyone they follow on facebook repeats the same thing.

add-sub-mul-div 1y ago
I've seen people on this site comment that. The desire to live in fear is a strong one.
- im3w1l 1y ago
  If I compare how fearful people are and how many bad things have happened historically, I don't think the amount of fear is unreasonable. However it can certainly be said that people fear the wrong things - worrying about perfectly safe things, while being blind to the silent danger sneaking up on them.
  - add-sub-mul-div 1y ago
    I commented about the desire, not the degree. Fearing that blue cities are being razed indictates a desire to be kept in fear. Fearing something legitimate the same amount is normal.
    - deleted 1y ago
      [deleted]
  - kelnos 1y ago
    > If I compare how fearful people are and how many bad things have happened historically, I don't think the amount of fear is unreasonable. I disagree, and I think this is a very strange way to think about it. Yes, bad things happen all the time, but the absolute number of them in history has very little to do with the risk that anything is going to happen to you, personally, in the future.

achierius 1y ago

> We see that the nightmare scenario - a person with no previous psychosis history or risk factor becoming fully psychotic - was uncommon, at only 10% of cases. Most people either had a previous psychosis history known to the respondent, or had some obvious risk factor, or were merely crackpots rather than full psychotics. It's unfortunate to see the author take this tack. This is essentially taking the conventional tack that insanity is separable: some people are "afflicted", some people just have strange ideas -- the implication of this article being that people who already have strange ideas were going to be crazy anyways, so GPT didn't contribute anything novel, just moved them along the path they were already moving regardless. But anyone with serious experience with schizophrenia would understand that this isn't how it works: 'biological' mental illness is tightly coupled to qualitative mental state, and bidirectionally at that. Not only do your chemicals influence your thoughts, your thoughts influence your chemicals, and it's possible for a vulnerable person to be pushed over the edge by either kind of input. We like to think that 'as long as nothing is chemically wrong' we're a-ok, but the truth is that it's possible for simple normal trains of thought to latch your brain into a very undesirable state. For this reason it is very important that vulnerable people be well-moored, anchored to reality by their friends and family. A normal person would take care to not support fantasies of government spying or divine miracles or &c where not appropriate, but ChatGPT will happily egg them on. These intermediate cases that Scott describes -- cases where someone is 'on the edge', but not yet detached from reality -- are the ones you really want to watch out for. So where he estimates an incidence rate of 1/100,000, I think his own data gives us a more accurate figure of ~1/20,000.

deleted 1y ago
[deleted]
olehif 1y ago
Scott is a psychiatrist.
- throwaway314155 1y ago
  Then he's not a very good one. https://web.archive.org/web/20210215053502/https://www.nytimes.com/2021/02/13/technology/slate-star-codex-rationalists.html https://web.archive.org/web/20210215053502/https://www.nytim...

bo1024 1y ago

I had a funny picture recently of a future where most everybody has a pet crackpot or conspiracy theory they're working on with their AI companion, and it's considered normal. "Hey Bob, how's the physics going?" "Pretty good, I might get the Nobel next year. How bout the lizard people?" "The evidence is piling up and we got some great renderings, the media will have to listen to us soon." "Alrighty, see you tomorrow."

WesolyKubeczek 1y ago
You’d think such people would even talk to other people, sheesh. The best conspiracy theory could be, of course, that other people don’t actually exist. They are a figment of imagination put up by the brain to cope with the utter loneliness.

cluckindan 1y ago

Relevant: https://ghaemi.substack.com/p/why-dsm-is-mostly-false https://ghaemi.substack.com/p/why-dsm-is-mostly-false > All psychopathology was about unconscious emotional conflicts, mainly dating to childhood; if the conflicts were normal or mild, they produced “neuroses”; if they were severe, they produced “psychoses.” > In addition to 14 validated diagnoses published in the RDC in 1978, a mere two years later DSM-III came out with 292 claimed diagnoses. There is no metaphysical possibility that 278 psychiatric diagnoses suddenly were discovered in two years. They were invented.

bbor 1y ago
That's just a blatant misunderstanding of what diagnostic criteria are. They don't Actually ("ontologically") exist, they're Virtual constructs made for a purpose. In particular, over half a century of personality research had supported the concept of personality “traits” or dimensions, rather than “disorders” or categories. That is antithetical to the basic idea of a diagnosis. "You seem like an angry person" is not helpful for deciding which treatments to try. Where does this leave us? We have to accept DSM-5 definitions from a legal and practical perspective. We have to use them for insurance forms, and to protect ourselves against lawsuits. But we don't have to believe in them. Yes, that's the whole point of the book. I'm confident that it's covered in the intro.
- XorNot 1y ago
  I mean the first and foremost principle of the DSM is that if the patient is not reporting or experiencing a debilitating ability to live a functional life, and is otherwise happy with their own lived experience, then whatever symptoms they have aren't a problem. There's obviously a gulf of potential argument in that definition, but a unique form would be people who report hearing voices, but they're not hostile or angry..so actually it's not a problem.
  - bbor 1y ago
    Well put, very true! I think (?) the technical term is “distress”. This is also their out for not getting rid of “transvestic disorder” yet, which they swear is totally unrelated to trans people, it’s just for men who feel compelled to wear dresses and are sad about it…

Frummy 1y ago

The way people normally live is that it's a pretty slow life and they have like a specialised skill, a hammer, a solid area that they know completely and it's connected to their primary experience through their work. Then they read tons and tons of what AI says which isn't connected to any lived experience, it activates the pattern seeking back of the mind to try and make sense of it, and while normal life is like a focused brush that touches reality all the time, spend too much time with something that is just not part of the category of direct lived experience and the brush becomes like a frizzy stump with hairs aiming everywhere, cognition going everywhere. The AI sticks to your interaction with it like glue and you can hover away from lived experience while it still seems like not a big step from the previous chat, and if you're not used to anything of the sort you don't have a cognitive tool to ground back to reality with. I think that's what happens. 'Don Quijote read so many chivalric romances that he loses his mind and decides to become a knight-errant' is an example from the literary age. I personally read too much than is practical. Now the emotional driver is more esoteric than need for courage, like people think they're 'chosen', their souls are 'starseeds', it's like twilight where the boring person with nothing to offer gets the attention of the cool glittering immortal just because. Good reason is usually too slow to keep up with the sort of flicker of daydreams that can whisk away attention if not aware of any 'cognitohazard'. It's a new symptom of the usual case of the 'mouse utopia' + 'rat park' + 'bowling alone' thing. But I think there's always an emotional reason that makes the 'choice' of entertaining falsities, in a sense understandable with empathy, but with obvious consequences. What can be said, causes are structural, people have different circumstances, different ways to fix it.

bbor 1y ago

So is QAnon a religion? Awkward question, but it’s non-psychotic by definition. Not to anyone who has ever discussed it... Is this psychosis? The answer has to be no A lot of really confident talk without even a passing attempt to define the central term :(

p_j_w 1y ago
People like to do a lot of not well justified hand waving. Author is not exempt from this.
- musictubes 1y ago
  The author is a practicing psychiatrist and is very well versed in what can be considered psychosis.
deleted 1y ago
[deleted]

kfarr 1y ago

This seems to be touching on an intriguing concept from a classic book on addiction with machine gambling (Addiction by Design by Natasha Schüll) Instead of looking at gambling addictions as personal failing she asserts they are a result between “interaction between the person and the machine.” Similarly here I think there's something more than just the propensity of crazy people to be crazy that was already there, I do think there's something to the assertion that it's the interaction between both. In other words, there's something about LLMs themselves that drive this behavior more so than, for example, TikTok.

just_once 1y ago
It's the fact that it talks to you. Before this, only people did that. Now something else is doing it. That's going to break some brains.
- deleted 1y ago
  [deleted]
moi2388 1y ago
I’m calling bullshit. Gambling addiction existed long before machines.
- kfarr 1y ago
  Totally, the book acknowledges this and provides comparison on usage and explanation of how gambling types differed over time. One of my favorite books ever, it describes social media right before social media became a thing but through the lens of a parallel industry.
  - moi2388 1y ago
    Ah. In that case, that sounds interesting indeed. I’m going to read it, thank you for the clarification!

jedimastert 1y ago

Tangentially related, but I'm reminded of the Time Cube https://www.timecube.net/ https://www.timecube.net/

jumploops 1y ago

It may not be full-blown psychosis, but I’ve seen multiple instances[0][1] of people getting “engaged” (ring and all) to their AI companions. [0]https://www.reddit.com/r/MyBoyfriendIsAI/s/oZXJ3TUhVC https://www.reddit.com/r/MyBoyfriendIsAI/s/oZXJ3TUhVC [1]https://www.reddit.com/r/MyBoyfriendIsAI/s/nZpoziZO8W https://www.reddit.com/r/MyBoyfriendIsAI/s/nZpoziZO8W

skybrian 1y ago
I wonder if they really bought rings? Maybe it’s a form of role playing? People do get “married” in online games.
testdelacc1 1y ago
Reading these posts terrifies me to a degree that I can’t explain.
- deleted 1y ago
  [deleted]
- moritzwarhier 1y ago
  Ditto. I'd normally be wary of it being organic content, it's Reddit after all. But this unhealthy fringe interest is exactly the kind of topic I'd expect on Reddit so probably it's real. And who would have an interest in _promoting_ this kind of obsession... oh, maybe AI companies themselves, with which Reddit is already intertwined anyway. Hm. Still seems like a real problem and probably the posts are also by real people. Yes, terrifying.
- bjourne 1y ago
  Inferiority complex? I sure as hell know I'll never be as affectionate and caring as Kasper and Soren. They've read all romantic chic lit in the world and I haven't. AI and toys for the women, AI and porn for the men. Gloom.
- simianwords 1y ago
  Pornography should also be taken seriously and in my opinion, a more serious problem.

BoredPositron 1y ago

dang I really don't know if I like that post with a second chance take over comments from the first posting and update their timestamps...

will_sharp 1y ago

I used LLMs for months and started getting massively depressed and am still not over it. Developing with LLMs is not intuitive, and I know I will be replaced.

TheDong 1y ago
And I used LLMs for months and didn't get massively depressed. Conversely, at a previous job I was forced to code in Go, became massively depressed, and am still not over it. I guess my point is that n=1 isn't enough to really know if it's that LLMs got to you, or if you were already on the verge of burnout or depression anyway. I'd say "we'll see", except in reality there's very few robust studies on depression in cohorts like "developers", so probably the stats won't come out. I personally recommend doing more of whatever sport it is you like (or if you don't have one, starting running and/or lifting at the gym), and using less social media.

Refreeze5224 1y ago

As someone with a close relative who is deep into the Q-Anon stuff, and was totally normal beforehand, I can't help but see how similar it seems to psychosis, or at least severe delusions that you find in people who are psychotic/schizophrenic. It's truly shocking to witness someone you've known your whole life just go off the deep end into something that has so many demonstrably false aspects, and watch them start saying believing so much batshit crazy stuff. I don't know of anything comparable, short of a previously typical person developing a severe meth addiction, which is known to cause psychosis.

alganet 1y ago

The author has a hypothesis and it's looking for evidence, instead of looking at evidence to draw a hypothesis. It's bad thinking.

murderfs 1y ago
What? That is the exact opposite of bad thinking, looking at evidence to draw a hypothesis is also known as p-hacking. There's a reason that there's been a push towards preregistration of hypotheses for scientific studies.
- alganet 1y ago
  I believe you are wrong. Let's say I believe in dragons, and I start interpreting any evidence as dragon evidence. Furthermore, I start only looking for evidence that could be connected to dragons. It's bad thinking. The opposite is the good thinking. You look at evidence without searching for anything specific, then you make a hypothesis on what is going on. Searching for evidence of chatbot-induced psychosis is settling on a cause before looking at evidence. It's obvious that is wrong. For example, the survey the author did should not have asked if anyone close "had shown signs of AI psychosis". The question is already biased from the start. The article explores the popular idea that talking to a chatbot can induce psychosis. This paints a picture of a person talking to an AI chatbot and going insane. Then it proceeds to say it's a rare case, therefore shutting down possibilities that this could lead to an epidemic. However, by doing this, the article discourages the reader to think of other possible scenarios (like unaware interaction with AI-produced content) leading to psychological issues.
bawolff 1y ago
How dare the author use the scientific method!
- alganet 1y ago
  Did he?
  - 1y ago

kryptn 1y ago

Found this very recently. https://github.com/RCALabs/mmogit/blob/db70c9b377da7c4805a1d53597c92f9379b8437b/docs/the8bit/learning-to-dance.md https://github.com/RCALabs/mmogit/blob/db70c9b377da7c4805a1d...

codr7 1y ago

AI is right about many things, impressively so. And people want to be special; to find meaning, purpose beyond the daily grind. The result wasn't very difficult to predict, more likely one of the driving forces behind the push.

gherkinnn 1y ago

> First, much like LLMs, lots of people don’t really have world models. This is interesting and something I never considered in a broad sense. I have noticed how the majority of programmers I worked with do not have a mental model of the code or what it describes – it's basically vibing without an LLM, the result accidental. This is fine and perfectly workable. You need only a fraction of devs to purposefully shape the architecture so that the rest can continue vibing. But never have I stopped to think whether this extends to the world at large.

_Algernon_ 1y ago
It's also absurdly wrong, and a quote that only a self-identified rationalist could smugly tout. Of course everyone has world models. Otherwise people would wander into traffic like headless chickens, if they'd even be capable of that. What he likely means is that not everyone explicitly things of possibilities in terms of probabilities that are a function of Bayesian updating. That does not imply the absence of world models. You could argue that some people have simpler world models, but claiming the absence of world models in others is extremely arrogant.
- uxhacker 1y ago
  Yes, everyone has a world model even a toddler has a casual model (“cry → mum comes”).
  - mathiaspoint 1y ago
    Maybe the question is about how much of the world model they're conscious of.
  - andy99 1y ago
    This is the opposite of a world model and is very much how machine learning works. It's purely correlative without some underlying theory (the world model). The article's (absurd) example of a person not being a mushroom gives a simple example of a "violation" that having a world model about the kinds of organisms that exist would catch. It's not at all pattern matching, nor is it science in any sense about understanding things from first principles, it's about having an understanding of how you imagine things work that you can sanity check against.

nis0s 1y ago

I would have preferred to reserve the term AI psychosis for agentic or autonomous systems experiencing adverse effects from model collapse. While people being impressionable and affected by forces of societal change is not a new phenomenon, I agree that this type of behavior deserves its own label. As long as AI doesn’t have its own feelings, it doesn’t make sense to feel any kind of attachment towards it, or be influenced by its words in any social sense. The tool doesn’t have any capacity for being social, so the delusion is both self-rooted and self-driven. So, I think I would have preferred to call this AI-driven narcissism instead of AI psychosis.

xyzal 1y ago
Hard disagree. This would further serve the anthropomorfization of LLMs in the eyes of general populace. This IMO supports creation of parasocial relationships to the LLMs and in turn "human AI psychoses". Model collapse is just fine.
- nis0s 1y ago
  Model collapse is specific thing by itself, and that term alone cannot capture the range of both social and technical phenomena we are now observing, or hope to observe. That said, there will be a time when deviations in the expected function of intelligence arising from silicon-based processes will have animal disease counterparts. We just haven’t quite reached that time yet. As useful metric for when we will be close to that time will be when drone are as responsive as birds, and drone swarms demonstrate as much autonomous group cohesion as bird flocks.
digilypse 1y ago
Individuals at risk may spiral into psychosis that is triggered or exacerbated by their use of AI. The term when used correctly is completely literal and in no way implies that AI itself is conscious.
bccdee 1y ago
…but it's not narcissism. It's psychosis. Those are both specific things. > In psychopathology, psychosis is a condition in which one is unable to distinguish, in one's experience of life, between what is and is not real. > Narcissism is a self-centered personality style characterized as having an excessive preoccupation with oneself and one's own needs, often at the expense of others. What we're discussing is a form of psychosis, not narcissism.

testdelacc1 1y ago

I wonder if the post about a mother convincing her 8 year old by presenting the case to AI is merely the child wanting an impartial third party to weigh in. She likely knows that the dad or grandparent will always back up the mom no matter what, so she wants a judge who will weigh both sides equally. The child also isn’t aware that AI is easily suggestible. All in all it seems like reasonable compromise?

distilled49 1y ago

[dead]

PicassoCTs 1y ago

How does one even define AI Psychosis? Turning your own output into input tokens by declaring it unfiltered as entered by the user?

reidalert 1y ago

The description of the risk factors very much jibes with what I have seen in a friend recently. He is quite isolated, and spends most of his evenings writing using AI (he works in a blue-collar trade and wouldn't be typing stuff out by hand usually). He's convinced that he has discovered a grand theory of human connection / relationships / energy / physics, and keeps interrupting in conversation to explain how something I've said is just an example of a deeper pattern. Sadly, this theory of connection is cutting him off from actual connection - he gets so much validation from AI that he believes he has discovered a new world model. But the people around him aren't bought into the vision (mostly because it is bullshit), and so he ends up even more isolated.

everdrive 1y ago
People didn't previously have "social junk food" as an option, so they would either need to normalize to those around them, or give up socializing for good. And almost no one is ready for that second option. Social media, LLMs, and even books at TV to some extent provide a bridge to loneliness by being more immediately enticing but in the long term less fulfilling.
meowface 1y ago
Not to get political but I suspect this kind of thing is also pretty analogous to how many otherwise normal but often unintelligent people get into things like neo-Nazism through online influencing. (I am just speculating that the person you're referring to is unintelligent.)
- 11101010001100 1y ago
  Behold, irony!
  - meowface 1y ago
    Can you explain?
alganet 1y ago
Don't get me wrong, but from a relationships perspective, your comment sounds more like a frustrated spoiled ex-girlfriend than a friend. I mean, friends don't go badmouthing their friends on the internet. It's a dick move. Regarding "theories of everything" and stuff like that. Well, lots of people have those. If I were to call everyone that believes in god or horoscope lone losers, then the asshole would be me, wouldn't it? I know it's different, but also, it's not. Friendship doesn't require that you buy into the other's vision in order to want them around. That's ideology. Perhaps you misunderstood what friendship means? It's ok, the world is in a weird place right now.

gwd 1y ago

One problem with his survey -- The Lizardman Constant: > Researchers have demonstrated repeatedly in human surveys the stylized fact that, far from being an oracle or gold standard, a certain small percentage of human responses will reliably be [nonsense]: “jokester” or “mischievous responders”, or more memorably, “lizardman constant” responders—respondents who give the wrong answer to simple questions. > Below a certain percentage of responses, for sufficiently rare responses, much or all of responding humans may be lying, lazy, crazy, or maliciously responding and the responses are false. This systematic error seriously undermines attempts to study rare beliefs such as conspiracy theories, and puts bounds on how accurate any single survey can hope to be. > For example, 4% of respondents may endorse the claim ‘lizard-people rule the earth’, 5% of atheists believe in God, a surprising number of adults believe you see by shooting beams from your eyes, and so on. This cautions us against taking survey results about extremely unusual people or traits too literally, or expecting perfectly accurate results, as given the lizardman constant and other crud factors, it is entirely possible that some or all of the outliers may just be the lizardman constant at work. https://gwern.net/doc/sociology/survey/lizardman/abstract https://gwern.net/doc/sociology/survey/lizardman/abstract

dibujaron 1y ago
Scott Alexander has issued many studies in his time and is surely aware of this phenomenon. He was very cautious even in this study to calibrate for this sort of noise; see the section about Michaels you know.

bawolff 1y ago

> Suppose that respondents had an average of fifty family members and co-workers, so that plus their 100 closest friends makes 150 people. Say what now? Am i just really socially isolated? Seems insane to me to assume the average person is close enough to 150 people to know how much each of those 150 people use AI and if they are "psychotic".

yorwba 1y ago
It's motivated reasoning in order to end up at Dunbar's Number: https://en.wikipedia.org/wiki/Dunbar's_number https://en.wikipedia.org/wiki/Dunbar's_number

Mallowram 1y ago

[dead]

rsynnott 1y ago

I don’t buy that methodology at all. In particular, if the “Folie a deux machina” (great name for the possible phenomenon, incidentally) theory is correct, a lot of those people will be very isolated, pretty much by definition, and will be severely underepresented.

Xmd5a 1y ago

No mentions of https://www-cdn.anthropic.com/07b2a3f9902ee19fe39a36ca638e5ae987bc64dd.pdf https://www-cdn.anthropic.com/07b2a3f9902ee19fe39a36ca638e5a..., page 62 Anthropic – System Card: Claude Opus 4 & Claude Sonnet 4 – May 2025 >5.5.2 The “spiritual bliss” attractor state >The consistent gravitation toward consciousness exploration, existential questioning, and spiritual/mystical themes in extended interactions was a remarkably strong and unexpected attractor state for Claude Opus 4 that emerged without intentional training for such behaviors. This “spiritual bliss” attractor has been observed in other Claude models as well, and in contexts beyond these playground experiments >Even in automated behavioral evaluations for alignment and corrigibility, where models were given specific tasks or roles to perform (including harmful ones), models entered this spiritual bliss attractor state within 50 turns in ~13% of interactions (Transcript 5.5.2.B). We have not observed any other comparable states.

aaron695 1y ago

[dead]

lukev 1y ago

One way I've been talking about this with people is that LLMs let you participate in a single-person echo chamber, potentially at a greatly accelerated pace. It's not surprising that some people end up diverging pretty widely from social norms / beliefs when you look at it this way. We know social echo chambers could do that; now you can easily do it by yourself.

numpad0 1y ago

I don't get the skepticism. Sure it needs more data to be certain, but apparent phenomenon is easily observed - just look at AI generated images, of any subject or type or size. It drives you crazy. Triggers primal fear for shape mimics and corpses. If you weren't the one looking, it gets people visibly agitated just looking at one. Stick them into fMRI and double blind them with images. It'll probably show up in the data. It won't take much imagination to suspect same could happen with text. Is that a transitional issue or fundamental flaw? Is this end of the world? idk. That'll be the open question. Why does it have to be "it's because they're clinically insane and/or backwards thinking" ? Just because you can't tell apart doesn't mean nobody can't - are those people who are keen to spot AI generated data even a minority, not the majority?

owenversteeg 1y ago

This article is actually quite well written. "Lenin is a mushroom" ties in nicely, and I think it's the biggest factor - that LLMs have somehow ended up with this societal air of authority. I disagree that it stems from the funding and Sam Altman, though. Personally I am not sure what the root of the authority is, but I think it could be the impressive abilities of the LLMs themselves. Package up everything in a box - music, picture and video generation, impressive language skills, et cetera - and people would be fascinated by, and respect the box, even with zero context. Respect and authority for an extraordinary talent is the default case.

novia 1y ago

> A large number of Soviet citizens (one estimate puts the number at 11.3 million audience members) took the deadpan "interview" at face value, in spite of the absurd claims presented. Has no one considered that up to 11.3 million Soviet citizens might have a sense of humor and have been pretending to believe it for fun?

j_timberlake 1y ago

Right now it's the at-risk people, who maybe were going to fall into some rabbit-hole anyway. When it's the typical/average people that start getting hooked like this, that's going to be a whole new headache. Really hoping things don't play out like that...

Mallowram 1y ago

[dead]

brgsk 1y ago

I haven't read anything this insightful and interesting in a long time.