11 ms·

It feels like I'm fighting uphill battle when it comes to bouncing ideas off of a model. I'll set things up in the context with instructions similar to. "Help m

It feels like I'm fighting uphill battle when it comes to bouncing ideas off of a model. I'll set things up in the context with instructions similar to. "Help me refine my ideas, challenge, push back, and don't just be agreeable." It works for a bit but eventually the conversation creeps back into complacency and syncophancy. I'll check it too by asking "are you just placating me?" the funny thing is that often it'll admit that, yes, it wasn't being very critical, and then procede to over correct and become a complete contrarian. and not in a way that's useful either. very frustrating. I've found that Opus 4.6 is worse about this than 4.5. 4.5 does a better job IMO of following instructions and not drifting into the mode where it acts like everything i say is a grand revelation from up high.

righthand 6mo ago
That’s because the model isn’t actually thinking, pushing back, and challenging your ideas. It’s just statistically agreeing with you until it reaches too wide of a context. You’re living in the delusion that it’s “working” or having a “conversation” with you.
- alehlopeh 6mo ago
  How is conceptualizing what the model is doing as having a conversation any different from any other abstraction? “No, the browser isn’t downloading a file. The electrons in the silicon are actually…”
  - colechristensen 6mo ago
    There are people with a philosophical objection to using everyday words to describe LLM interactions for various reasons, but commonly because they're worried stupid people will confuse the LLM for a person. Which, I suppose stupid people will do that, but I'm not inventing a parallel language or putting a * next to each thing which means "this, but with an LLM instead of a person"
    - cruffle_duffle 6mo ago
      That is an interesting way of looking at that, thanks for the perspective! Like, the words fit… why create a second parallel language for describing LLM behavior. Somebody else said it… the whole “it’s a stochastic parrot” thing is sooooo cliche and boring at this point. It’s like, duh… what is your point?
  - righthand 6mo ago
    One is an abstraction, another is a fantasy.
cyanydeez 6mo ago
So, there's things you're fighting against when trying to constrain the behavior of the llm. First, those beginning instructions are being quickly ignored as the longer context changes the probabilities. After every round, it get pushed into whatever context you drive towards. The fix is chopping out that context and providing it before each new round. something like `<rules><question><answer>` -> `<question><answer><rules><question>`. This would always preface your question with your prefered rules and remove those rules from the end of the context. The reason why this isn't done is because it poisons the KV cache, and doing that causes the cloud companies to spin up more inference.
margalabargala 6mo ago
Considering 4.6 came with a ton of changes around tooling and prompting this isn't terribly surprising.
magicalhippo 6mo ago
Gemini seems to be fairly good at keeping the custom instructions in mind. In mine I've told it to not assume my ideas are good and provide critique where appropriate. And I find it does that fairly well.
- lelanthran 6mo ago
  > Gemini seems to be fairly good at keeping the custom instructions in mind. Unless those instructions are "stop providing links to you for every question ".
  - magicalhippo 6mo ago
    I think that's part of the RAG pipeline, at least to me it looks to be separate from the model output. Models are notorious for getting URLs wrong so makes sense to me to handle it separately.
- steve_adams_86 6mo ago
  Same. This works fine for Claude in my experience. My user prompt is fairly large and encourages certain behaviours I want to see, which involves being critical and considering the strengths and weaknesses of ideas before drawing conclusions. As someone else mentioned, there does seem to be a phenomenon where saying DO NOT DO X causes a sort of attention bias on X which can lead to X occurring despite the clear instructions. I've never empirically tested that, I've just noticed better results over the years when telling it what paths to stick to rather than specific things not do to.
  - koverstreet 6mo ago
    That happens with humans too :) It's why positive feedback that draws attention to the behavior you want to encourage often works better. "Attention" is lower level and more fundamental than reasoning by syllogism.
  - magicalhippo 6mo ago
    > there does seem to be a phenomenon where saying DO NOT DO X causes a sort of attention bias on X which can lead to X occurring despite the clear instructions It's a thing with people too[1], ie do not think about a white bear. [1]: https://en.wikipedia.org/wiki/Ironic_process_theory https://en.wikipedia.org/wiki/Ironic_process_theory
Loughla 6mo ago
That's because you need actual logic and thought to be able to decide when to be critical and when to agree. Chatbots can't do that. They can only predict what comes next statistically. So, I guess you're asking if the average Internet comment agrees with you or not. I'm not sure there's much value there. Chatbots are good at tasks (make this pdf an accessible word document or sort the data by x), not decision making.
- kvirani 6mo ago
  I'm not convinced that "actual logic and thought" aren't just about inferring what comes next statistically based on experience.
  - plagiarist 6mo ago
    Then the machines still need a more sophisticated "experience" compared to what they have currently.
  - righthand 6mo ago
    Communicating is usually about inferring. I dont think token to token. And I don’t think “well statistically I could say ‘and’ next but I will say ‘also’ instead to give my speech some flash”. If I decided on swapping a word I would have made my decision long ago, not in the moment. Thought and logic are not me pouring through my brain finding a statistical path to any answer. Often I stop and say “I dont know”.
  - Swizec 6mo ago
    > I'm not convinced that "actual logic and thought" aren't just about inferring what comes next statistically based on experience. Often they are the exact opposite. Entire fields of math and science talk about this. Causation vs correlation, confirmation bias, base rate fallacy, bayesian reasoning, sharp shooter fallacy, etc. All of those were developed because “inferring from experience” leads you to the wrong conclusion.
    - theptip 6mo ago
secret_agent 6mo ago
Use positive requests for behavior. For some reason, counter prompts "Don't do X" seems to put more attention on X than the "Don't do." It's something like target fixation, "Oh shit I don't want to hit that pothole..." bang
- ambicapter 6mo ago
  This is a well known problem in these kind of systems. I’m not 100% on what the issue is mechanically but it’s something like they can only represent the existence of things and not non-existence so you end up with a sort of “don’t think of the pink elephant” type of problem.
  - SpicyLemonZest 6mo ago
    Isn't it just that, in the underlying text distribution, both "X" and "don't do X" are positively correlated with the subsequent presence of X? I've never seen that analysis run directly but it would surprise me if it weren't true.
RugnirViking 6mo ago
check out this article that was posted here a while back https://www.randalolson.com/2026/02/07/the-are-you-sure-problem-why-your-ai-keeps-changing-its-mind/ https://www.randalolson.com/2026/02/07/the-are-you-sure-prob... The article's main idea is that for an AI, sycophancy or adversarial (contrarian) are the two available modes only. It's because they don't have enough context to make defensible decisions. You need to include a bunch of fuzzy stuff around the situation, far more than it strictly "needs" to help it stick to its guns and actually make decisions confidently I think this is interesting as an idea. I do find that when I give really detailed context about my team, other teams, ours and their okrs, goals, things I know people like or are passionate about, it gives better answers and is more confident. but its also often wrong, or overindexes on these things I have written. In practise, its very difficult to get enough of this on paper without a: holding a frankly worrying level of sensitive information (is it a good idea to write down what I really think of various people's weaknesses and strengths?) and b: spending hours each day merely establishing ongoing context of what I heard at lunch or who's off sick today or whatever, plus I know that research shows longer context can degrade performance, so in theory you want to somehow cut it down to only that which truly matters for the task at hand and and and... goodness gracious its all very time consuming and im not sure its worth the squeeze
- cruffle_duffle 6mo ago
  > goodness gracious its all very time consuming and im not sure its worth the squeeze And when you step back you start to wonder if all you are doing is trying to get the model to echo what you already know in your gut back to you.
- awithrow 6mo ago
  oh that's great. thanks for the link!
- oldfrenchfries 6mo ago
  This is great, thanks for sharing!
dkersten 6mo ago
I find Kimi white good if you ask it for critical feedback. It’s BRUTAL but offers solutions.
- ohyoutravel 6mo ago
  Not soft, not mild, but BRUTAL! This broke my brain!
- awithrow 6mo ago
  what is Kimi white?
  - dkersten 6mo ago
    I was typing quickly on my phone. I meant “quite”, “I find Kimi quite good”
dinkumthinkum 6mo ago
You're not wrong and you're not crazy. In fact, you are absolutely right! It is not just These things are not just casual enablers. They are full-on palace sycophants following the naked emperor showering him with praise for his sartorial elegance. /s
rsynnott 6mo ago
Why not... do this with a person, instead? Other humans are available. (Seriously, I don't understand this. Plenty of humans will be only too happy to argue with you.)
- awithrow 6mo ago
  oh i do as well. I think of the LLM as another tool in the toolbox, not a replacement for interactions. There is something different about having a rubber duck as a service though.
- kelseyfrog 6mo ago
  "the percentage of U.S. adults who report having no close friends has quadrupled to 12% since 1990"[1] 1. https://www.happiness.hks.harvard.edu/february-2025-issue/the-friendship-recession-the-lost-art-of-connecting https://www.happiness.hks.harvard.edu/february-2025-issue/th...
  - nathan_compton 6mo ago
    More technology is probably the solution to this!
- layla5alive 6mo ago
  Many other humans are .... Not very available - certainly many shut down when conversations reach a certain level of depth or require great focus or introspection..
  - balamatom 6mo ago
    Depth? Introspection? I'd say these days the norm is to not simply shut down, but to become irrevocably and insidiously hostile, the moment someone hints at the existence of such a thing as "ground truth", "subjective interpretation", "being right or wrong" - or any of the bits and bobs that might lead one to discover the proper scary notion, "consensus reality". "What do you mean social reality is a constructed by the consensus of the participants? Reality is what has been drilled into my head under threat of starvation! How dare you exist!", et cetera. You've heard it translated into Business English countless times. They are deathly afraid of becoming aware of their own conditioned state of teleological illiteracy - i.e. how they are trained to know what they are doing, but never why they are doing it. It's especially bad with the guys who cosplay US STEM gang. One is not permitted a position of significance in this world without receiving this conditioning, and I figure it's precisely this global state of cognitive disavowal which props up the value of the US dollar - and all sorts of other standees you might've recently interacted with as if they're not 2D cutouts (metaphorical ones! metaphorical!). PSA: Look up "locus of control" and "double bind". Between those two, you might be able to get a glimpse of what's going on - but have some sort of non-addictive sedative handy in case you do.
raincole 6mo ago
My rule of thumb: 1. Only one shot or two shot. Never try to have a prolonged conversation with an LLM. 2. Give specific numbers. Like "give me two alternative libraries" or "tell me three possible ways this might fail."
post-it 6mo ago
> I'll check it too by asking "are you just placating me?" the funny thing is that often it'll admit that, yes, it wasn't being very critical, and then procede to over correct and become a complete contrarian. and not in a way that's useful either. It's not admitting anything. Your question diverts it down a path where it acts the part of a former sycophant who is now being critical, because that question is now upstream of its current state. Never make the mistake of asking an LLM about its intentions. It doesn't have any intentions, but your question will alter its behaviour.
- godelski 6mo ago
  > Your question diverts it down a path where it acts the part of a former sycophant who is now being critical I think people really have a hard time understanding a sycophant can be contrarian. But a yesman can say yes by saying no https://news.ycombinator.com/item?id=47484664 https://news.ycombinator.com/item?id=47484664
  - cheesecompiler 6mo ago
    > a yesman can say yes by saying no What a great way to summarize LLM behaviour in 2026
    - noisy_boy 6mo ago
      >> a yesman can say yes by saying no >What a great way to summarize LLM behaviour in 2026 Well they have been trained on words spoken by humans and that has been a human behaviour since time immemorial. E.g.: "I do not agree with you that you were wrong. I do apologize for my strong disagreement but we actually do need your continued guidance desperately."
- layer8 6mo ago
  I think “admit” here is just a description of what the LLM was saying. It doesn’t imply that the OP thinks the LLM has internal beliefs matching that.
ajkjk 6mo ago
'admit' isn't really the right word for that... the fact that it was placating you wasn't true until you prompted it to say so. Unlike a person who has an 'internal emotional state' independent of what they say that you can probe by asking questions.
- awithrow 6mo ago
  'admit' is anthropomorphizing the behavior, sure. The point is that sometimes the model's response will tighten, flag things that were overly supportive or what not. Sometimes it wont, it'll state that previous positions are still supported and continue to press it. Its not like either response is 'correct' but it can alter the rest of the responses in ways that are useful.
colechristensen 6mo ago
>"Help me refine my ideas, challenge, push back, and don't just be agreeable." This is where you're doing it wrong. If your LLM has a problem being more agreeable than you want, prompt it in a way that makes being agreeable contrary to your real intentions. "there are bugs and logic problems in this code" "find the strongest refutation of this argument" "I don't like this plan and need to develop a solid argument against it" Asking for top ten lists is a good method, it will rarely not come up with anything but you can go back and forth and refine until it's 10 ten reasons why your plan is bad are all insubstantial nonsense then you've made progress
Forgeties79 6mo ago
I usually put “do not praise me, do not use emojis, I just want straight answers” something along those lines and it’s been surprisingly effective. Though it helps I can’t run particularly heavy duty models/don't carry on the “conversation” for super long durations.
anandram27 6mo ago
Could be an aspect of eval awareness mb
GuB-42 6mo ago
I find the best way is to give the LLM as little information as possible about where you want to go. For example don't say "I think pineapple pizzas are the best, am I right?", say "What is the general consensus on pineapple pizzas?".
arjie 6mo ago
I haven't found that work at all. But the way I usually frame it for this is that I present it as someone else's work that I disagree with (sometimes "this is a junior engineer's work and he often does dumb shit" as an introduction to an essay). In this way, it tries to be agreeable with me and can rationalize various opposition. And then I can select from that.
jimmyjazz14 6mo ago
Yeah, I have never had good results with refining ideas with models or really any interactions with models outside of rote task such as coding or analyzing document structures, I don't know why I was ever surprised by this as its obvious that LLMs just aren't capable of original thinking. I think part of the problem is that these things were marketed originally as chatbots when that is honestly their weakest use-case. I think even when I was expressly try to not anthropomorphize LLMs I still sorta did in early days, but the less I do so the more utility I get from them.
sinuhe69 6mo ago
The longer you converse with the LLM, the more frequent the same keywords will occur and Thu’s they will only strengthen the probabilistic choices of the LLM. The LLM do not exactly follow the instructions like a program would do, nor does it truly understand its role or what its “said”. It looks fine on first tries and short texts, but will drift more the longer the conversation goes. The only way the LLM can maintain the appearance of a normal conversation is to repeatedly feed it with the users requests. We can see it with the shell construct the AI company built around their chat interface.

righthand 6mo ago

That’s because the model isn’t actually thinking, pushing back, and challenging your ideas. It’s just statistically agreeing with you until it reaches too wide of a context. You’re living in the delusion that it’s “working” or having a “conversation” with you.

alehlopeh 6mo ago
How is conceptualizing what the model is doing as having a conversation any different from any other abstraction? “No, the browser isn’t downloading a file. The electrons in the silicon are actually…”
- colechristensen 6mo ago
  There are people with a philosophical objection to using everyday words to describe LLM interactions for various reasons, but commonly because they're worried stupid people will confuse the LLM for a person. Which, I suppose stupid people will do that, but I'm not inventing a parallel language or putting a * next to each thing which means "this, but with an LLM instead of a person"
  - cruffle_duffle 6mo ago
    That is an interesting way of looking at that, thanks for the perspective! Like, the words fit… why create a second parallel language for describing LLM behavior. Somebody else said it… the whole “it’s a stochastic parrot” thing is sooooo cliche and boring at this point. It’s like, duh… what is your point?
- righthand 6mo ago
  One is an abstraction, another is a fantasy.

cyanydeez 6mo ago

So, there's things you're fighting against when trying to constrain the behavior of the llm. First, those beginning instructions are being quickly ignored as the longer context changes the probabilities. After every round, it get pushed into whatever context you drive towards. The fix is chopping out that context and providing it before each new round. something like `<rules><question><answer>` -> `<question><answer><rules><question>`. This would always preface your question with your prefered rules and remove those rules from the end of the context. The reason why this isn't done is because it poisons the KV cache, and doing that causes the cloud companies to spin up more inference.

margalabargala 6mo ago

Considering 4.6 came with a ton of changes around tooling and prompting this isn't terribly surprising.

magicalhippo 6mo ago

Gemini seems to be fairly good at keeping the custom instructions in mind. In mine I've told it to not assume my ideas are good and provide critique where appropriate. And I find it does that fairly well.

lelanthran 6mo ago
> Gemini seems to be fairly good at keeping the custom instructions in mind. Unless those instructions are "stop providing links to you for every question ".
- magicalhippo 6mo ago
  I think that's part of the RAG pipeline, at least to me it looks to be separate from the model output. Models are notorious for getting URLs wrong so makes sense to me to handle it separately.
steve_adams_86 6mo ago
Same. This works fine for Claude in my experience. My user prompt is fairly large and encourages certain behaviours I want to see, which involves being critical and considering the strengths and weaknesses of ideas before drawing conclusions. As someone else mentioned, there does seem to be a phenomenon where saying DO NOT DO X causes a sort of attention bias on X which can lead to X occurring despite the clear instructions. I've never empirically tested that, I've just noticed better results over the years when telling it what paths to stick to rather than specific things not do to.
- koverstreet 6mo ago
  That happens with humans too :) It's why positive feedback that draws attention to the behavior you want to encourage often works better. "Attention" is lower level and more fundamental than reasoning by syllogism.
- magicalhippo 6mo ago
  > there does seem to be a phenomenon where saying DO NOT DO X causes a sort of attention bias on X which can lead to X occurring despite the clear instructions It's a thing with people too[1], ie do not think about a white bear. [1]: https://en.wikipedia.org/wiki/Ironic_process_theory https://en.wikipedia.org/wiki/Ironic_process_theory

Loughla 6mo ago

That's because you need actual logic and thought to be able to decide when to be critical and when to agree. Chatbots can't do that. They can only predict what comes next statistically. So, I guess you're asking if the average Internet comment agrees with you or not. I'm not sure there's much value there. Chatbots are good at tasks (make this pdf an accessible word document or sort the data by x), not decision making.

kvirani 6mo ago
I'm not convinced that "actual logic and thought" aren't just about inferring what comes next statistically based on experience.
- plagiarist 6mo ago
  Then the machines still need a more sophisticated "experience" compared to what they have currently.
- righthand 6mo ago
  Communicating is usually about inferring. I dont think token to token. And I don’t think “well statistically I could say ‘and’ next but I will say ‘also’ instead to give my speech some flash”. If I decided on swapping a word I would have made my decision long ago, not in the moment. Thought and logic are not me pouring through my brain finding a statistical path to any answer. Often I stop and say “I dont know”.
- Swizec 6mo ago
  > I'm not convinced that "actual logic and thought" aren't just about inferring what comes next statistically based on experience. Often they are the exact opposite. Entire fields of math and science talk about this. Causation vs correlation, confirmation bias, base rate fallacy, bayesian reasoning, sharp shooter fallacy, etc. All of those were developed because “inferring from experience” leads you to the wrong conclusion.
  - theptip 6mo ago

secret_agent 6mo ago

Use positive requests for behavior. For some reason, counter prompts "Don't do X" seems to put more attention on X than the "Don't do." It's something like target fixation, "Oh shit I don't want to hit that pothole..." bang

ambicapter 6mo ago
This is a well known problem in these kind of systems. I’m not 100% on what the issue is mechanically but it’s something like they can only represent the existence of things and not non-existence so you end up with a sort of “don’t think of the pink elephant” type of problem.
- SpicyLemonZest 6mo ago
  Isn't it just that, in the underlying text distribution, both "X" and "don't do X" are positively correlated with the subsequent presence of X? I've never seen that analysis run directly but it would surprise me if it weren't true.

RugnirViking 6mo ago

check out this article that was posted here a while back https://www.randalolson.com/2026/02/07/the-are-you-sure-problem-why-your-ai-keeps-changing-its-mind/ https://www.randalolson.com/2026/02/07/the-are-you-sure-prob... The article's main idea is that for an AI, sycophancy or adversarial (contrarian) are the two available modes only. It's because they don't have enough context to make defensible decisions. You need to include a bunch of fuzzy stuff around the situation, far more than it strictly "needs" to help it stick to its guns and actually make decisions confidently I think this is interesting as an idea. I do find that when I give really detailed context about my team, other teams, ours and their okrs, goals, things I know people like or are passionate about, it gives better answers and is more confident. but its also often wrong, or overindexes on these things I have written. In practise, its very difficult to get enough of this on paper without a: holding a frankly worrying level of sensitive information (is it a good idea to write down what I really think of various people's weaknesses and strengths?) and b: spending hours each day merely establishing ongoing context of what I heard at lunch or who's off sick today or whatever, plus I know that research shows longer context can degrade performance, so in theory you want to somehow cut it down to only that which truly matters for the task at hand and and and... goodness gracious its all very time consuming and im not sure its worth the squeeze

cruffle_duffle 6mo ago
> goodness gracious its all very time consuming and im not sure its worth the squeeze And when you step back you start to wonder if all you are doing is trying to get the model to echo what you already know in your gut back to you.
awithrow 6mo ago
oh that's great. thanks for the link!
oldfrenchfries 6mo ago
This is great, thanks for sharing!

dkersten 6mo ago

I find Kimi white good if you ask it for critical feedback. It’s BRUTAL but offers solutions.

ohyoutravel 6mo ago
Not soft, not mild, but BRUTAL! This broke my brain!
awithrow 6mo ago
what is Kimi white?
- dkersten 6mo ago
  I was typing quickly on my phone. I meant “quite”, “I find Kimi quite good”

dinkumthinkum 6mo ago

You're not wrong and you're not crazy. In fact, you are absolutely right! It is not just These things are not just casual enablers. They are full-on palace sycophants following the naked emperor showering him with praise for his sartorial elegance. /s

rsynnott 6mo ago

Why not... do this with a person, instead? Other humans are available. (Seriously, I don't understand this. Plenty of humans will be only too happy to argue with you.)

awithrow 6mo ago
oh i do as well. I think of the LLM as another tool in the toolbox, not a replacement for interactions. There is something different about having a rubber duck as a service though.
kelseyfrog 6mo ago
"the percentage of U.S. adults who report having no close friends has quadrupled to 12% since 1990"[1] 1. https://www.happiness.hks.harvard.edu/february-2025-issue/the-friendship-recession-the-lost-art-of-connecting https://www.happiness.hks.harvard.edu/february-2025-issue/th...
- nathan_compton 6mo ago
  More technology is probably the solution to this!
layla5alive 6mo ago
Many other humans are .... Not very available - certainly many shut down when conversations reach a certain level of depth or require great focus or introspection..
- balamatom 6mo ago
  Depth? Introspection? I'd say these days the norm is to not simply shut down, but to become irrevocably and insidiously hostile, the moment someone hints at the existence of such a thing as "ground truth", "subjective interpretation", "being right or wrong" - or any of the bits and bobs that might lead one to discover the proper scary notion, "consensus reality". "What do you mean social reality is a constructed by the consensus of the participants? Reality is what has been drilled into my head under threat of starvation! How dare you exist!", et cetera. You've heard it translated into Business English countless times. They are deathly afraid of becoming aware of their own conditioned state of teleological illiteracy - i.e. how they are trained to know what they are doing, but never why they are doing it. It's especially bad with the guys who cosplay US STEM gang. One is not permitted a position of significance in this world without receiving this conditioning, and I figure it's precisely this global state of cognitive disavowal which props up the value of the US dollar - and all sorts of other standees you might've recently interacted with as if they're not 2D cutouts (metaphorical ones! metaphorical!). PSA: Look up "locus of control" and "double bind". Between those two, you might be able to get a glimpse of what's going on - but have some sort of non-addictive sedative handy in case you do.

raincole 6mo ago

My rule of thumb: 1. Only one shot or two shot. Never try to have a prolonged conversation with an LLM. 2. Give specific numbers. Like "give me two alternative libraries" or "tell me three possible ways this might fail."

post-it 6mo ago

> I'll check it too by asking "are you just placating me?" the funny thing is that often it'll admit that, yes, it wasn't being very critical, and then procede to over correct and become a complete contrarian. and not in a way that's useful either. It's not admitting anything. Your question diverts it down a path where it acts the part of a former sycophant who is now being critical, because that question is now upstream of its current state. Never make the mistake of asking an LLM about its intentions. It doesn't have any intentions, but your question will alter its behaviour.

godelski 6mo ago
> Your question diverts it down a path where it acts the part of a former sycophant who is now being critical I think people really have a hard time understanding a sycophant can be contrarian. But a yesman can say yes by saying no https://news.ycombinator.com/item?id=47484664 https://news.ycombinator.com/item?id=47484664
- cheesecompiler 6mo ago
  > a yesman can say yes by saying no What a great way to summarize LLM behaviour in 2026
  - noisy_boy 6mo ago
    >> a yesman can say yes by saying no >What a great way to summarize LLM behaviour in 2026 Well they have been trained on words spoken by humans and that has been a human behaviour since time immemorial. E.g.: "I do not agree with you that you were wrong. I do apologize for my strong disagreement but we actually do need your continued guidance desperately."
layer8 6mo ago
I think “admit” here is just a description of what the LLM was saying. It doesn’t imply that the OP thinks the LLM has internal beliefs matching that.

ajkjk 6mo ago

'admit' isn't really the right word for that... the fact that it was placating you wasn't true until you prompted it to say so. Unlike a person who has an 'internal emotional state' independent of what they say that you can probe by asking questions.

awithrow 6mo ago
'admit' is anthropomorphizing the behavior, sure. The point is that sometimes the model's response will tighten, flag things that were overly supportive or what not. Sometimes it wont, it'll state that previous positions are still supported and continue to press it. Its not like either response is 'correct' but it can alter the rest of the responses in ways that are useful.

colechristensen 6mo ago

>"Help me refine my ideas, challenge, push back, and don't just be agreeable." This is where you're doing it wrong. If your LLM has a problem being more agreeable than you want, prompt it in a way that makes being agreeable contrary to your real intentions. "there are bugs and logic problems in this code" "find the strongest refutation of this argument" "I don't like this plan and need to develop a solid argument against it" Asking for top ten lists is a good method, it will rarely not come up with anything but you can go back and forth and refine until it's 10 ten reasons why your plan is bad are all insubstantial nonsense then you've made progress

Forgeties79 6mo ago

I usually put “do not praise me, do not use emojis, I just want straight answers” something along those lines and it’s been surprisingly effective. Though it helps I can’t run particularly heavy duty models/don't carry on the “conversation” for super long durations.

anandram27 6mo ago

Could be an aspect of eval awareness mb

GuB-42 6mo ago

I find the best way is to give the LLM as little information as possible about where you want to go. For example don't say "I think pineapple pizzas are the best, am I right?", say "What is the general consensus on pineapple pizzas?".

arjie 6mo ago

I haven't found that work at all. But the way I usually frame it for this is that I present it as someone else's work that I disagree with (sometimes "this is a junior engineer's work and he often does dumb shit" as an introduction to an essay). In this way, it tries to be agreeable with me and can rationalize various opposition. And then I can select from that.

jimmyjazz14 6mo ago

Yeah, I have never had good results with refining ideas with models or really any interactions with models outside of rote task such as coding or analyzing document structures, I don't know why I was ever surprised by this as its obvious that LLMs just aren't capable of original thinking. I think part of the problem is that these things were marketed originally as chatbots when that is honestly their weakest use-case. I think even when I was expressly try to not anthropomorphize LLMs I still sorta did in early days, but the less I do so the more utility I get from them.

sinuhe69 6mo ago

The longer you converse with the LLM, the more frequent the same keywords will occur and Thu’s they will only strengthen the probabilistic choices of the LLM. The LLM do not exactly follow the instructions like a program would do, nor does it truly understand its role or what its “said”. It looks fine on first tries and short texts, but will drift more the longer the conversation goes. The only way the LLM can maintain the appearance of a normal conversation is to repeatedly feed it with the users requests. We can see it with the shell construct the AI company built around their chat interface.