7 ms·
Pretty sure Anthropic and other providers prepend these "official" system prompts to your conversation even if you send in a custom system prompt otherwise it w
by swingboy 1mo ago
Pretty sure Anthropic and other providers prepend these "official" system prompts to your conversation even if you send in a custom system prompt otherwise it would be trivial to produce CSAM, etc.
- fullmoon 1mo agoI don’t think so. If you start a new Claude Code session without a system prompt, it doesn’t even know what model it is and hallucinates being some old variant of Sonnet.
- flaburgan 1mo agoHow do you start a session without a system prompt if you use ACP in Zed for example?
- marcelo-earth 1mo agoIt makes sense. But how do you start a Claude Code session without a system prompt?
- tosh 1mo agoat least according to their documentation they do not afaiu they have other systems for denying and re-routing requests
- DANmode 1mo agoThey use non-LLM gates for this. Otherwise DANmode and similar jailbreaks would still be as easily accessible as they were at the beginning.
- LPisGood 1mo agoThe system prompt is (and cannot be) the only guardrail against things like that, because any system prompt is little more than a good suggestion.
- CodesInChaos 1mo agoI wouldn't put auch limitations in the system prompt. A mix of fine-tuning and out-of-band detection appears to be a better fit.
- ardel95 1mo agoCSAM, and other harms, are typically detected using a set of specially trained, faster and cheaper models (and out of band matching techniques) that run before and after the main model. Any mention in the system prompt is mostly defense in depth, and to make refusals more graceful.
- whstl 1mo agoAlso, the system prompt, or even something reinforced on every message, is nowhere near as strong as its internal training or as an external safeguard. If the prompt were the only protection, it would be extremely easy to produce illegal content after a long session.