8 ms·
This article argues that there's no reliable way to detect prompt injection: https://simonwillison.net/2022/Sep/17/prompt-injection-more-ai/ https://simonwillis
by alangpierce 3y ago
This article argues that there's no reliable way to detect prompt injection:
https://simonwillison.net/2022/Sep/17/prompt-injection-more-ai/ https://simonwillison.net/2022/Sep/17/prompt-injection-more-...
One solution to some indirect prompt injection attacks is proposed in this article, where you "sandbox" untrusted content into a second LLM that isn't given the ability to decide which actions to take:
https://simonwillison.net/2023/Apr/25/dual-llm-pattern/ https://simonwillison.net/2023/Apr/25/dual-llm-pattern/
- SkyPuncher 3y agoI see absolutely no way prompt injection can be fully protected against. There are nearly infinite ways to word an attack. You can only protect against the most common of them.
- cubefox 3y agoWhat about this approach? https://news.ycombinator.com/item?id=35929145 https://news.ycombinator.com/item?id=35929145
- SkyPuncher 3y agoI mean, sure that'd work, but doesn't it defeat most of the point in using an LLM? The only way that works is if you escape _all_ user content. If you're telling an LLM to ignore all user content, then why are you using an LLM in the first place?
- cubefox 3y agoThe approach isn't to ignore all "user" content at all. It is trained to follow instructions in normal text; only instructions contained in specially quoted text (that is, external text, like a website) are ignored. Quotation would apply to Bing's search abilities or ChatGPTs new Browsing Mode, which both load website content into the context window.