When Context Misleads: Intent-Guided Decoding for Robust Retrieval-Augmented Generation
2026-08-17 • Computation and Language
Computation and LanguageArtificial Intelligence
AI summaryⓘ
The authors address a problem in retrieval-augmented generation (RAG) where the system must decide how much to trust external information it brings in. They propose Intent-Guided Decoding (IGD), which adjusts how the model weighs retrieved evidence versus its own learned knowledge based on what the user wants. IGD improves factual accuracy, especially when external information conflicts or may mislead. Tests on various benchmarks show IGD helps models better balance using evidence and sticking to facts.
Retrieval-Augmented GenerationLarge Language ModelsParametric MemoryUser IntentAnswer-Level FilteringToken-Level CorrectionFactualityFaithfulnessQuestion Answering BenchmarksContext-Following
Authors
Haolin Jin, Pengyue Yang, Huaming Chen
Abstract
Retrieval-augmented generation (RAG) improves large language models by grounding generation in external evidence, but it also introduces a source trust problem: retrieved context may be useful, irrelevant, or even misleading. Existing RAG systems often apply a fixed trust policy toward retrieved evidence, which can either over-trust incorrect context or underuse context when the user explicitly asks for context-following behavior. Therefore, we propose Intent-Guided Decoding (IGD), a framework that arbitrates between retrieved context and parametric memory according to user intent. IGD uses answer-level filtering and token-level correction to steer the final decoding trajectory between retrieved context and parametric memory. We evaluate IGD on three faithful QA benchmarks and three factual-conflict benchmarks across five LLMs, IGD substantially improves factual recovery, achieving gains of up to 65.4 percentage points on factual-conflict benchmarks over Direct RAG, while preserving or improving strict context-following behavior, this findings highlight the importance of balancing factuality and faithfulness in RAG.