论文精选

Google 论文:RAG 投毒只需复述用户查询即可操纵检索结果

精选理由

Google 这篇论文发现假文档只要复述用户问题就能被 RAG 检索到,5 条文档就能让排序质量掉四成,做 RAG 的都该看看怎么防。

Google 的一项研究指出,RAG 系统的投毒攻击主要靠在假文档中复述用户查询文本,是否附加攻击指令几乎不影响文档是否被检索。实验在 623 条查询上测试了关键词检索器、语义检索器和 BGE-small 嵌入模型,每条查询注入最多 5 条复述文档。仅约语料 0.1% 的 5 条文档,就让经典检索器的排序质量下降 40.2% 到 61.5%。用过滤规则拒绝复述查询的文档虽能拦截攻击,但在 NFCorpus 数据集上把干净查询的排序质量从 0.267 砍半到 0.134。

原文 · rohanpaul_ai

New Google paper on RAG poisoning.

Shows that poisoning a RAG system mostly depends on copying the query, since adding an attack payload barely changed which documents got retrieved.

fake documents only need to repeat a user’s question to be picked up by AI systems that search before answering, and adding hidden attack instructions barely changed that.

RAG systems pull documents from sources that outsiders can sometimes write to. Before any hidden instruction can act, the poisoned page first has to get past the retriever.

For each test query, they added up to 5 documents repeating its text, with or without an attack payload. They tested classic keyword and semantic retrievers plus the dense embedding model BGE-small on 623 queries.

Just 5 documents, about 0.1% of the corpus, cut classic retrievers' ranking quality by 40.2% to 61.5%. A filter that rejects query-echoing documents blocked them but halved clean ranking quality on NFCorpus, from 0.267 to 0.134.

If your RAG index accepts outside content, flag documents that mirror user queries, and measure any filter's cost on clean queries too.