5 ms·
Are LLMs easy to "dupe"? E.g. by placing provably wrong "information" somewhere on the internet? If so what countermeasures can help?
by aleph-z 1mo ago
Are LLMs easy to "dupe"? E.g. by placing provably wrong "information" somewhere on the internet? If so what countermeasures can help?
- thisislife2 1mo ago- Companies Are Using Reddit to Manipulate ChatGPT and Google AI Search - https://www.404media.co/companies-are-using-reddit-to-manipulate-chatgpt-and-google-ai-search/ https://www.404media.co/companies-are-using-reddit-to-manipu... - AI Search Has a Misinformation Problem — and DuckDuckGo Just Proved It - https://www.informertech.com/post/duckduckgo-reddit-misinformation-campaign https://www.informertech.com/post/duckduckgo-reddit-misinfor...
- hazrmard 1mo agoI read this paper (https://arxiv.org/pdf/2410.13722 https://arxiv.org/pdf/2410.13722) on pre-training poisoning of LLMs. The authors demonstrated that poisoning 0.1% of the pretraining dataset can measurably manipulate beliefs of the model. What I read about is a more sophisticated supply chain attack. But these days, search previews use retrieval-augmented generation to parse websites to generate answers on the fly. In that case you don't need to poison the supply chain; good SEO is enough.