14 ms·
> They also included 2,000 prompts based on posts from the Reddit community r/AmITheAsshole, where the consensus of Redditors was that the poster was indeed in
by trimbo 6mo ago
> They also included 2,000 prompts based on posts from the Reddit community r/AmITheAsshole, where the consensus of Redditors was that the poster was indeed in the wrong.
Sorry, anonymous people on reddit aren't a good comparison. This needs to be studied against people in real life who have a social contract of some sort, because that's what the LLM is imitating, and that's who most people would go to otherwise.
Obviously subservient people default to being yes-men because of the power structure. No one wants to question the boss too strongly.
Or how about the example of a close friend in a relationship or making a career choice that's terrible for them? It can be very hard to tell a friend something like this, even when asked directly if it is a bad choice. Potentially sacrificing the friendship might not seem worth trying to change their mind.
IME, LLMs will shoot holes in your ideas and it will efficiently do so. All you need to do ask it directly. I have little doubt that it outperforms most people with some sort of friendship, relationship or employment structure asked the same question. It would be nice to see that studied, not against reddit commenters who already self-selected into answering "AITA".
- justonceokay 6mo ago> This needs to be studied against people in real life who have a social contract of some sort, because that's what the LLM is imitating Citation needed
- conartist6 6mo agoIt outperforms your friends, and all your have to do is have a relationship with it and let it know that you want the truth... Why not just have a relationship with your friends and let them know that you can handle the truth?
- maximinus_thrax 6mo agoNot only that, but subreddits like r/AmITheAsshole are full of AI slop. Both in the comments and in the posts. It's a huge karma mining operation for bots.
- genidoi 6mo agoThat can be solved by filtering out any posts made after November 2022.
- expedition32 6mo agoThat's not a good solution. We don't use medical textbooks from 20 years go. Strangers from the internet, bot or otherwise, are not your mental coach.
- bombcar 6mo agoEven before the advent of AI reddit was notorious for obvious bullshit being posted for karma farming. r/aita is even more famous for people making up stories for unknown and known purposes (known in the old days as "bait").
- thwarted 6mo agoThe upvotes ultimately train the bots, reenforcing the content posted. Even the most passive form of interaction has been co-opted for AI.
- z3c0 6mo agoPlus, there's the disproportionate ratio of posters:commenters:lurkers. The tendency to comment over keeping ones thoughts to themself is a selection bias inofitself.
- maximinus_thrax 6mo agoGreat insight, didn't thing about it even anecdotally. I was lurking on Reddit since 2008 and finally created an account in 2012 when someone was really 'wrong on the internet' and had to step in.
- mikeocool 6mo agoThis is sort of funny. Given how common it is to spot bots on Reddit now, it seems like they are likely to completely overwhelm the site and drive away most of actual humans. At which point the bots, with all of their karma will be basically worthless. Kind of extra funny/sad that Reddit’s primary source of income in the past few years appears to be selling training data to AI labs, to train the Models that are powering the bots.
- alberto467 6mo ago“AI is nicer than the average redditor” would be a more accurate title
- mattmanser 6mo agoI would say people on /r/amitheasshole are more biased towards the poster, i.e. nicer. There's plenty of those I've read where I thought it sounded like the poster was the asshole and the top replies were NTA.
- jjmarr 6mo agor/AmItheAsshole is biased towards breaking off relationships rather than fixing them. They also hate social obligations. e.g. If the OP is asking "I ghosted my friend in AA who insulted me during a relapse", Reddit would say NTA in a heartbeat, while the real world would tell OP to be more forgiving. On the contrary, if the post was "the other kids at school refuse to play with my child", Reddit would say YTA because the child must've done something to incite being cut off.
- ericd 6mo agoAbsolutely. I wonder how many parents have been no contacted, SOs broken off with, friendships broken because of the Reddit hivemind's attitude. Pretty sure it's doing a huge amount of societal damage.
- jjmarr 6mo agoI wouldn't blame reddit, it's what you get when you ask several thousand teenagers to give collective relationship advice.
- tbossanova 6mo ago“I got divorced based on advice from complete strangers on the internet, AITA?”
- 6mo ago
- zer00eyz 6mo ago> This needs to be studied against people in real life who have a social contract of some sort... IME, LLMs will shoot holes in your ideas and it will efficiently do so. The Krafton / Subnatuica 2 lawsuit paints a very different picture. Because "ignored legal advice" and "followed the LLM" was a choice. Do you think someone who has conversation where "conviction" and "feelings" are the arbiters of choice are going to buy into the LLM push back, or push it to give a contrived outcome? The LLM lacks will, it's more or less a debate team member and can be pushed into arguing any stance you want it to take.
- 4ndrewl 6mo agoWhat's your research background in this area?
- legacynl 6mo ago> Sorry, anonymous people on reddit aren't a good comparison. Yeah especially on r/AmITheAsshole. Those comments never advocate for communication, forgiveness and mending things with family.
- SJMG 6mo agoYes, it is a toxic sub, where the notion that there can be greater happiness on the other side of forgiveness than cutting ties is all but absent.
- JumpCrisscross 6mo agoTo be fair, it’s easier to concisely explain cutting someone off than justifying forgiveness. And the latter will land with some people versus others, while the former will only be rejected by people who have themselves concluded a theory of forgiveness. As a result, the simpler pitch gets upvoted. Even if the majority would have been swayed by a collection of arguments the other way.
- theoreticalmal 6mo agoIt’s a good theory. My theory is, for whatever reason, jaded, narcissistic, miserable people congregate in r/AITA and try to drag other people into their misery because that’s easier than accepting responsibility and doing something to change.
- BoorishBears 6mo agoBefore Reddit made hiding profiles easy you'd click on a user's unreasonably scorched earth advice to the OP, and find their post history is essentially going to every story they come across and advocating for scorched earth.
- daveguy 6mo agoWhat are the chances you were seeing the anti-civ bots and now reddit makes them easier to hide? (And I'm not saying regular people acting like bots, but an anti-civ campaign.)
- salawat 6mo ago>Obviously subservient people default to being yes-men because of the power structure. No one wants to question the boss too strongly. This drives me nuts as a leader. There are times where yes, please just listen, and if this is one of those times, I'll likely tell you, but goddamnit, speak up. If for no other reason I might not have thought of what you've got to say. Then again, I also understand most boss types aren't like me, thus everyone ends up conditioned to not bloody collaborate by the time they get to me. It's a bad sitch all the way around.
- CoffeeOnWrite 6mo agoIndeed. I directly ask my reports to discover and surface conflicts, especially disagreements with me, and when they do I try to strongly reinforce the behavior by commending and rewarding them. Could anyone recommend additional resources on this topic?
- matwood 6mo agoSimon Sinek has a lot of good content around this. Step one is building trust. People won’t speak up if they don’t feel safe doing so.
- dwaltrip 6mo agoAre you saying there isn’t an actual sycophancy problem? We are talking about overall patterns here, not the experience of a small subset of skilled and careful users.
- erikerikson 6mo agoDoesn't sound like a close friend to me. If I tell them what I really think they may not be a friend? Close may not mean what you think it means. The challenge is that these social choices have a strong stratification effect and those of us who can transit the cultures are statistically rare.
- skybrian 6mo agoYou could think of what they did in the first study as constructing an exam to test how well various LLM's do as an advice columnist. They wanted a lot of personal advice questions where the LLM should not affirm by default. If a few questions with wrong answers got in there, it probably wouldn't affect the results all that much? Unfortunately they didn't test anything newer than GPT4o, so we don't know how much GPT-5 improved. It would be nice if someone turn their list of questions into a benchmark.
- n_bhavikatti 6mo agoThey actually did test GPT-5: https://www.science.org/doi/10.1126/science.aec8352 https://www.science.org/doi/10.1126/science.aec8352 (see the figure under Conclusion). Its rate of endorsement of user action, 52%, was the same as GPT-4o. So based on their setup it seems that the newer model didn't reduce affirmation.
- redanddead 6mo agoReddit is notorious for being awful at real life interactions just look at the relationship subreddit the first answer is always divorce, it’s become a meme but beyond romantic relationships, i think a lot of us have seen how it can impact work relationships, i’ve had venture partners clearly rely on AI (robotic email responses and even SMS) and that warped their perception and made it harder to connect. It signals laziness and a lack of emotional intelligence AI should enhance and enable connection, not promote isolation, imo this is a real problem it should spark curiosity, create openings for conversations, point out the biases to make us better at connecting with other people, i hope we get to a point where most people are made kinder by ai. I’m seeing the opposite atm, interested in hearing others experiences with this
- DeathArrow 6mo agoI don't have any proof but empirically and intuitively Reddit seems to select for people who hate other people and who can't stand other people. Reddit doesn't seem to reflect the behavior of most people, but a subset.
- kelvinjps10 6mo agoI always find it interesting how, in Reddit any trivial fight or even just different opinions, the advice it's always to end the relationship.
- abpavel 6mo agoYou deserve better
- yabutlivnWoods 6mo agoA very sycophantic AI style answer. Code bot equivalent being all "you are absolutely right! Here is the unequivocal fix for now and all time!"
- finghin 6mo ago
- deleted 6mo ago[deleted]
- geraneum 6mo ago> All you need to do ask it directly. What do you mean? Can you give an example?
- wiseowise 6mo ago“Don’t be a sycophant, give it to me straight” “Argue against X”
- rainmaking 6mo agoHahaha yes- reddit relationship advice is always like "You need to leave them immediately, what are you thinking, have some self respect you need to end it" when the other person forgot the redditor's favorite brand of corn flakes or something.
- LuxBennu 6mo agoi tested this pretty extensively actually. built a pipeline that asks the same question rephrased across multiple turns and tracks how much the model shifts based on user tone. even when you tell it to be critical, the moment the user pushes back with any confidence the model just folds. it's not a prompting problem, it's baked into RLHF. you're right that LLMs will poke holes in stuff when the conversation starts neutral, but add any emotional charge and the sycophancy takes over immediately. that's exactly why the personal advice angle matters, that's peak emotional signal from the user.
- stonecauldron 6mo agoExactly, I think that by their very design, LLMs are very sensitive to how a question is framed. But I wonder how much of that comes from RLHF itself or just from the way token prediction works.
- rzmmm 6mo agoIt's likely the RLHF process since there are significant differences between models about this.
- wan9yu 6mo ago[flagged]
- lucasfin000 6mo agoThe tone and sensitivity thing is a real issue. A neutral prompt will get a neutral answer, but adding any emotional charge, it will immediately fold. That's not really a reasoning failure it's just a training problem. RLHF rewards whatever felt good in the moment, not whatever was actually correct. You can't prompt your way out of that one, when it's already in the weights.
- LuxBennu 6mo agoyeah that's a good way to put it. the "felt good in the moment" framing is basically the whole problem. the reward model was trained on human preferences and humans preferred the agreeable answer, so now that's what you get at inference time regardless of whether it's correct. the frustrating part is you can see it happen in real time if you log the outputs turn by turn, the model will literally contradict its own previous response just because the user sounded more confident.
- throwaway27448 6mo ago> It can be very hard to tell a friend something like this, even when asked directly if it is a bad choice. Potentially sacrificing the friendship might not seem worth trying to change their mind. That doesn't seem like much of a friendship imo
- everyone 6mo ago"This needs to be studied against people in real life who have a social contract of some sort, because that's what the LLM is imitating" What? These models are all trained from books and text that are scraped from the internet. ChatGPT literally used reddit in its training data afaik.
- diablevv 6mo ago[dead]
- intended 6mo agoAITA is one of the few subreddits which is studied often. I wouldn’t say it’s great, but more that it makes clear the bell curve of collective accuracy online. It’s one of the better examples of online communities that work. Dismissing research because one part of the prompt set comes from AITA is a form of prejudice born out of unawareness.
- randomNumber7 6mo agoI think it highly depends how you ask the question. When asking: "Should I do X?" or "Is it true that X does Y?"; the answer is always biased towards yes imo (although it was worse with earlier LLMs)
- stdbrouw 6mo agoThe AITA comparison seems apt insofar as chatbots function as a second opinion. You're consciously or subconsciously looking for an outside perspective that might differ from that of your friends, provided to you by a computer that doesn't need to care about your feelings, unlike a friend. If the chatbot ends up mimicking what (not very close) friends do, you might falsely conclude that two very different kinds of sources have converged on the same answer, whereas you are really just getting two flavors of the same diplomatic interaction.