6 ms·
Super cool that this works. I'm surprised these companies re-use the same encryption key across models! I wonder if you can use these for attacks, like this pr
by x312 1mo ago
Super cool that this works. I'm surprised these companies re-use the same encryption key across models!
I wonder if you can use these for attacks, like this previous paper showing that if you know how a model reasons, you can "fake its thinking" to control it? https://news.ycombinator.com/item?id=48631888 https://news.ycombinator.com/item?id=48631888
- yubblegum 1mo agoSeriously, what does it take to encrypt per session? There are many ways to make it scalable and efficient so I am wondering if this is left like this to allow interested 3rd parties ahem unobtrusively peek what people are doing with the AI. (Thanks for the link. That’s an interesting idea!)
- dannyw 1mo agoThe provider has the hidden text anyway; this isn’t customer managed encryption.
- yubblegum 1mo agoSure, but if each session has a unique key then these need to be managed and stored and unauthorized access to these leaves tracks. So all that had to be 'compromised' is a single universally applicable key. Again, the question stands: session based encryption can be scalable and efficient. Why aren't they using it?
- paxys 1mo agoThe exploit here isn’t a leaked encryption key. It’s pretty likely that they are already using a unique key per conversation. The raw CoT eventually reaches the model, and you can convince the model to share it with you.
- theapadayo 1mo agoYeah encryption isn't the issue. The only way I see to fix this is if you stop the user from switching models mid-session, or strip out the thoughts when switching models. Either way you're degrading the user experience.
- yubblegum 1mo agoIf a different model is using encyrpted blocks of another model, then by definition it is no longer a session scoped bit of information. Since you can give it to any other session and another model, clearly it doesn't even have to be the same user. Therefore, there is only one (set) of universally available key(s) used by all models across all sessions.
- flexagoon 1mo ago> I'm surprised these companies re-use the same encryption key across models I assume switching the model in the middle of a conversation is intended behavior (very useful in coding agents, for example)