InceptionRAG attack breaks defenses in retrieval augmented generation systems
InceptionRAG: Stealthy Poisoning Attack Against Retrieval-Augmented Generation
Cryptography and Security
Summary
Retrieval-augmented generation (RAG) systems use external documents to help answer questions more accurately. The authors found that current defenses cannot stop a new kind of attack where harmful information is split across multiple harmless-looking parts. When these parts are combined during retrieval, the language model mistakenly reasons to arrive at wrong conclusions. They also created a new defense method to isolate these tricky chains of information to prevent such attacks.
What this means in practice
- •For ai security teams: Detect and defend against stealthy poisoning attacks that exploit multi-document reasoning in RAG systems.
- •For enterprise ai platform engineers: Implement document isolation techniques to reduce vulnerabilities of RAG-enhanced language models in production environments.
Authors
Jiachang Zhang, Min Chen, Xiao Ren, Zhenyong Zhang, Yuanchao Shu, Yunjun Gao, Zhikun Zhang
Abstract
Retrieval-augmented generation (RAG) systems enhance large language models (LLMs) with external knowledge but have been demonstrated to be vulnerable to corpus poisoning. Existing poisoning attacks against RAG largely focus on single-point explicit injection, where the malicious payload is fully encapsulated within a single document. Consequently, recent mitigation mechanisms have evolved to identify and diminish these threats effectively. In this paper, we first verify that existing mitigation mechanisms are insufficient for a new class of threats: indirect logic induction. Motivated by this observation, we introduce InceptionRAG, a stealthy attack mechanism that subverts the standard attack paradigm. Instead of injecting explicit malicious payloads, InceptionRAG fragments it into a chain of dormant passages. These passages appear harmless and can bypass existing mitigation mechanisms when examined separately. However, when retrieved together, they trigger LLMs to self-deduce target misinformation via multi-hop reasoning. To further improve the applicability of InceptionRAG in black-box settings, we propose zeroth-order suffix optimization (ZOSO) to automate the generation of authoritative suffixes. Extensive evaluations across three datasets and three LLMs demonstrate that InceptionRAG achieves an attack success rate exceeding 80% even under rigorous adversarial constraints. In particular, InceptionRAG shows superior evasion capabilities, effectively bypassing established defenses that mitigate traditional single-document injections. Our findings expose a concerning paradox: the stronger reasoning capabilities of LLMs increase their vulnerability to reasoning-based poisoning attacks. To mitigate potential misuse, we propose a document isolation-based defense, HODOR, which decouples adversarial logical dependencies.