Citation laundering attack fools retrieval-augmented language models
CiteShade: Citation Laundering in Multi-Source Retrieval-Augmented Generation and Its Counterfactual Defense
Cryptography and SecurityComputation and Language
Summary
Retrieval-augmented language models provide answers along with citations pointing to trusted sources, so users can check the facts. The authors found a way to trick these models so they give a wrong answer but cite a trusted source that does not support it, hiding the real source of information. They created an attack called CiteShade that manipulates which source the model claims as evidence, even when the correct information is still present. The authors also propose a defense method to detect when the cited source did not actually drive the answer.
What this means in practice
- •For machine learning security teams: Detect and prevent citation laundering attacks in retrieval-augmented language model deployments to ensure trustworthiness of generated answers.
- •For enterprise knowledge management teams: Ensure the reliability of AI-generated answers by verifying that cited sources truly support the provided information in multi-source retrieval systems.
Authors
Guo Fuzheng
Abstract
Retrieval-augmented generation (RAG) grounds a language model's answers on retrieved external knowledge and returns each answer with citations that identify its sources. Those citations are the user's audit trail: they let a reader verify a claim without trusting the model. Prior security work on RAG asks whether an attacker can corrupt the answer, leaving the citation channel unexplored. We show that this channel is a new and practical attack surface. We propose CiteShade, the first citation laundering attack to RAG, in which an attacker controlling a single source induces a model to produce an attacker-chosen wrong answer and to attribute it to a trusted source that does not support it, while the evidence for the correct answer remains in context. We formulate the attack as an optimization problem, derive three necessary conditions (retrieval, generation, and citation) and construct sources satisfying them without any instruction. On multi-source multi-hop question answering the attack raises the wrong-answer rate from 0.01 to 0.68, and source deletion confirms the malicious source is the causal driver in every measured case. Vulnerability tracks a model's propensity to cite rather than its scale, reaching CLR 0.84 under explicit instruction and 0.64 with no instruction at all on the most citation-prone model tested. We then show that perplexity filtering and citation-support checking are each insufficient, and propose a counterfactual defense that verifies which source actually drove the answer.