Causally guided retrieval agent improves question answering over knowledge graphs

CARA: Causally Guided Retrieval Agent for Question Answering

Artificial IntelligenceInformation Retrieval

Summary

Finding the right information to answer a question in large text-rich knowledge graphs is tricky because just matching words isn’t enough. The authors created CARA, a system that checks if adding or removing pieces of information actually helps or hurts getting the answer, focusing on what really matters. CARA uses these checks to explore data more smartly and combines different searches for stronger results. When tested, CARA found better evidence more efficiently and helped produce more accurate answers. This shows that thinking about cause-and-effect helps machines find and use information better.

What this means in practice

  • For enterprise search teams: Improve enterprise knowledge search tools by focusing retrieval on causally relevant data to provide more accurate answers from interconnected documents and data points.
  • For technical support bots: Enhance automated support systems to find and verify critical troubleshooting information more efficiently using causal evidence evaluation over technical knowledge bases.

Authors

Weicheng Liu, Xiangcheng Wu, Depeng Xu, Haopeng Zhang, Xi Niu

Abstract

Retrieval-augmented generation over text-rich knowledge graphs requires identifying evidence that contributes to answering a question, rather than merely matching its textual content. Existing agentic retrievers adaptively balance global search and multi-hop exploration but primarily rely on textual similarity to guide retrieval. We propose CARA, a training-free, CAusally guided Retrieval Agent that explicitly evaluates the causal relevance of retrieved evidence through two complementary criteria: consequentiality, measuring whether adding evidence improves answerability, and essentiality, assessing whether removing evidence weakens answerability. CARA integrates these assessments with path consistency to guide graph exploration and employs reciprocal rank fusion to aggregate evidence from multiple agents. We evaluate CARA on three STaRK datasets and augment STaRK with ground-truth answers to enable downstream question answering evaluation. Experimental results demonstrate that CARA consistently outperforms existing retrievers. CARA also improves answer quality and faithfulness while examining fewer graph nodes and requiring fewer retrieval operations. These findings demonstrate that causal relevance assessment enables more focused and effective agentic graph retrieval, improving both retrieval performance and downstream question answering.