Language model resolves knowledge conflicts using prior information
Don't Pick a Winner: Knowledge Conflicts Need a Prior, Not a Rule
Artificial IntelligenceComputation and Language
Summary
When a language model answers questions using both its own memory and extra information it retrieves, these two sources can disagree. Existing methods pick one source to trust based on fixed rules, but these rules often make mistakes depending on the situation. The authors show that to decide which source to believe, it helps to have a way to estimate how often the extra information is correct and to evaluate each source's reliability separately. They build a system that does this and test it on a challenging benchmark, achieving much better accuracy than previous methods.
What this means in practice
- •For software engineers: Improve AI chatbots to better handle conflicting information from stored data and external sources by using a calibrated confidence system with prior knowledge of source accuracy.
- •For knowledge management teams: Enhance automated fact-checking tools to decide when retrieved external data should override or be combined with internal knowledge bases, ensuring greater answer reliability.
Authors
Minh Pham Van, Cong Tran
Abstract
A retrieval-augmented language model answers from two sources, the retrieved context and its own knowledge, and the two disagree whenever the context is incomplete or wrong. Existing methods settle the disagreement with a rule fixed at design time: follow the context, amplify it in proportion to how far it moves the model, or trust the model's memory. Each rule is confidently wrong in some deployment. In a decision-theoretic model of two sources, the score of any fixed rule is linear in the share of reliable contexts while the best attainable score is convex, so every fixed rule is beaten wherever it is not exactly optimal; and a symmetric divergence between the two sources, the signal adaptive decoding uses, carries no information about which one is right. Resolving a conflict therefore needs two inputs that no existing method keeps: evidence about each source separately, and a prior on how often the context is right. We test these claims on GRACE, a benchmark that intervenes on the knowledge-graph context of a fixed question, so that what the model knows, what the context says and what is true are measured separately. There, every baseline from four families is strong on right contexts or on wrong ones, never on both. We then build the two inputs: the model states its knowledge as notes, a small policy voices each note with a calibrated confidence, a reader answers in two voices, and a selector commits or abstains under a prior $π$ applied as an exact prior shift. With $π$ set to the true share of right contexts, the system is above the best baseline at every mixture of GRACE contexts, by up to 26 points. Its overall score is 45.3 and 35.0 (Qwen3.5-35B-A3B, Llama-3.1-8B), against 29.4 and 27.5 for the best baseline. On preregistered ClashEval it beats same-backbone CK-PLUG and TCR by 12 and 26 points. Where the model lacks the knowledge, the gains vanish, as the theory predicts.