6 ms·
Relevant anecdata because I've burned many a Claude sessions on this. If you're using Claude Code, then it's in the harness. At the close of many sessions, I w
by hysan 8d ago
Relevant anecdata because I've burned many a Claude sessions on this.
If you're using Claude Code, then it's in the harness. At the close of many sessions, I would start a meta conversation over why the LLM would consistently break certain rules. What it found when debugging itself is that some of the "contradicting" rules that I had were in fact, not from my rules. Instead, the instructions from its own harness had phrases telling it to do things like that. When something contradicts, its own instructions would outweigh any custom ones you write. Every rule variant I had tested (including the one that says it overrides the harness instructions - and yes, I've actually tested all the ideas in your comment too) has ultimately been unsuccessful due to this according to the LLM.
- vzmax 8d agoYou can't trust it's account on why it did something, it does not "remember". It will just make up something plausible sounding.
- mcv 8d agoTo me it explained its inability to follow my preferred coding style by blaming its training data.
- MichaelDickens 8d agoIt has access to its own instructions, right? Otherwise how would the instructions work?
- alienchow 8d agoAll instructions to LLMs are merely suggestions to nudge it in the right behaviour. Unless you have a deterministic guardrail that guards against a single specific action, everything else is a slot machine that's biased strongly in your favour. My global CLAUDE.md explicitly states "When commenting on code and configs, or writing MD files, strictly write within the domain of the content being commented on. DO NOT include information, negatives or ramblings from work sessions. For e.g. if commenting on a proto string field that is replacing an int field, do not comment that 'this is not an int field'". This reduced the idiocy of the agent (Opus 5 included) when writing documents. But I'm still catching it writing README.md talking about the negatives that it removed. Those belong in the memory if it is actually that important (most of the time it's junk), but Claude doesn't seem to understand and never ever learns.
- hysan 8d agoTrue which is why I said anecdata. But the point is that it’s a useful signal when it’s across enough sessions where it quotes back contradictions to you. If you’re willing to burn the tokens, the first thing it does is use a tool call to search your rules to see what it quoted wasn’t there. That leads to the LLM claiming two common conclusions (sometimes after a lot of back and forth) - it’s in the harness or it’s in the model. Over the 30+ sessions where I’ve tried this, the overwhelming claim was some variant of a harness instruction. It might not be there but given the consistency and how all my rule tests have failed in the same way that others describe, it at least makes it reasonable to conclude that it’s baked in somewhere and in a way where agent rules aren’t able to strongly affect the behavior. My bet would be on the harness because the class of undesired behavior follows that of a structured response.
- sillyfluke 8d agoMeta: > True which is why I said anecdata Forgive me for being cheeky, but presumably this is not why you said anecdata. You presumably said anecdata because you were describing your own personal sessions with Claude. Your original comment is written as someone who is assuming the AI is doing a deep deterministic analysis of its own internal systems in order to respond to you. It could simply be aping some discussion on the same topic within its training data, which if sensible is, as you say, not entirely worthless...
- hysan 7d agoFair. I should have reigned in my emotions and written that first comment with a lot less conviction and not jumped to my conclusion. It’s a topic I’ve been spending a lot of time learning about and got a bit too excited when a relevant thread popped up where I could chime in.
- pkulak 8d agoIt can read its own context. That’s how this all works.
- scrollaway 8d agoI know this is a popular position and it makes sense at face value when you think of LLMs as autocomplete systems. But it’s genuinely wrong. Relevant reading is most notably anthropic’s research on the J-space. LLMs will plan ahead of time helped with CoT, get to a plan and “store” it in j-space, and execute on that plan which means they can in fact “backtrack” and give you reasoning on why they did something, because it IS part of their state.
- catlifeonmars 8d agoIs there a way to dump the state directly?
- scrollaway 7d agoNot unless you are anthropic/openai, or run your own models.
- orbital-decay 7d agoJ-space is just one convenient projection (out of many) to look at these well known phenomena
- prawn 8d agoI've had Claude completely invent a word, and then admit under questioning that it was because it seemed plausible.
- ffsm8 8d ago[flagged]
- scrollaway 8d agoYour HN comment is not useful. If you think it is, then you're suffering from HN psychosis.
- eth0up 7d ago>you're suffering from HN psychosis. No. They must be hallucinating. And mind how 'they' so often select anthropomorphic terms, eg "hallucination" themselves, then ooze from the woodwork to behead someone who does the same, as if they own English and context itself. AI Psychosis, I think, simply means Double Standards and insulation from accountability in design. There is a very interesting Plausible Deniability angle here. But I'll leave that for the future.
- ffsm8 8d ago[flagged]
- scrollaway 8d agoAh yes. It's always the ones you most expect who come up with these retorts.
- ffsm8 8d ago[flagged]
- zaphirplane 8d ago100% sentences like i convinced it or made it admit are just weird But how else to describe the situation
- dostick 8d agoDoes the harness apply when you sue Claude model in third party IDE? like Opencode, or third party adds their harness on top, not replacing it?
- mixedmath 7d agoClaude models seem to work far better in Anthropic harnesses. I don't know if this is because they are trained in that context or because they offer different models through external APIs.
- laruss5 8d agoDepends what you mean by "apply." The harness-specific instructions (Claude Code's own system prompt quirks) only get sent if Claude Code is the program calling the API. OpenCode or any other third-party tool writes and sends its own system prompt instead, so those exact lines aren't there. Whatever behavior shows up across different harnesses anyway is more likely a trained default of the model itself, not something coming from a particular harness's prompt.
- drob518 8d agoThis is one of the reasons I use Pi. Pi’s minimal system prompt avoids contradiction between what the harness writer thinks is best and what the user thinks is best. The user specifies what the user wants and that’s pretty much the end of it.
- dmos62 8d agoI have found that asking the agent why it did X or Y is an essential part of iterating on its harness. It was non-obvious to me, probably because I'm used to having to reverse-engineer why a system does something, but that's not usually necessary with LLMs, unless the harness is complicated and you have to hunt for the instruction it tells you about.
- lucas_t_a 8d agoyou can always use /rewind to go back, always work thinking "can i shorten or clear my context now? and how should i do it?"