7 ms·

I feel like there is an easy solution here. Don’t even try. The LLM should only be trained on and have access to data and actions which the user is already app

I feel like there is an easy solution here. Don’t even try. The LLM should only be trained on and have access to data and actions which the user is already approved to have. Guaranteeing LLMs won’t ever be able to be prompted to do any certain thing is monstrously difficult and possibly impossible with current architectures. LLMs have tremendous potential but this limitation has to be negated architecturally for any deployment in the context of secure systems to be successful.

oakhill 3y ago
Access to data isn't enough - the data itself has to be trusted. In the OP the user had access to the google doc as it was shared with them but that doc isn't trusted because they didn't write it. Other examples could include a user uploading a PDF or document that came that includes content from an external source. Anytime a product injects data into prompts automatically is at risk of that data containing a malicious prompt. So there needs to be trusted input, limited scope in the output action, and in some cases user review of the output before an action is taken place. Trouble is that it's hard to evaluate when an input is trusted.
deleted 3y ago
[deleted]