GuardRAG defends AI answers from poisoned retrieval documents
GuardRAG: A Post-Retrieval Defense against Poisoning Attacks in Retrieval-Augmented Generation
Cryptography and Security
Summary
Retrieval-Augmented Generation (RAG) helps AI models get better answers by looking up information from outside sources. But bad actors can trick these models by putting false information into those sources, making the AI produce wrong answers. The authors designed GuardRAG, a tool that checks the group of documents retrieved for an answer and removes suspicious clusters that seem faulty or false. This makes the AI less likely to be fooled and still gives good answers when no attacks happen.
What this means in practice
- •For ai system engineers: Reduce the risk of poisoned evidence misleading AI answers in retrieval-augmented systems by filtering suspicious document groups after retrieval.
- •For enterprise search teams: Improve the reliability of knowledge-base search results by automatically detecting and removing coordinated false documents in retrieval results.
Authors
Linghui Jiao, Kang Wei, Fushuo Huo, Ruijie Zhao, Yiyang Ni, Jingcai Guo, Yipeng Zhou, Jun Li
Abstract
Retrieval-Augmented Generation (RAG) enhances the factual grounding of Large Language Models (LLMs) by incorporating external evidence, but also exposes them to corpus poisoning attacks. In particular, coordinated attacks can introduce multiple mutually reinforcing poisoned documents into retrieved contexts, making reliable evidence filtering challenging. To address this problem, we propose GuardRAG, a training-free and plug-and-play post-retrieval defense that exploits the semantic structure of retrieved evidence to identify suspicious document groups. Specifically, GuardRAG first expands the retrieval range and removes near-duplicate candidates, then partitions the retained documents using Hierarchical Agglomerative Clustering (HAC) with a query-adaptive threshold. Finally, it scores and prunes suspicious clusters based on query relevance, cluster size, and semantic compactness. Experiments across three question-answering datasets, multiple poisoning attacks, and six LLMs demonstrate that GuardRAG substantially reduces attack success rates while maintaining answer utility under clean retrieval.