CiteGuard-RAG improves answer trustworthiness by validating citations in AI systems
CiteGuard-RAG: A Validation-Centered AI System for Evidence-Grounded Question Answering
Computation and LanguageArtificial IntelligenceInformation Retrieval
Summary
Sometimes AI systems that find and answer questions from lots of information make up answers or use wrong sources. The researchers created CiteGuard-RAG, a system that checks the evidence and sources while it answers questions, making sure the answers are accurate and properly cited. They tested it on legal and privacy topics, showing it can almost always find the right information and confirm it before answering. Without these checks, the answers become less accurate, proving that validation is important for trust.
What this means in practice
- •For legal technology teams: Build AI tools that provide legally reliable answers supported by validated citations in housing and contract law scenarios.
- •For privacy compliance teams: Create systems that answer privacy-related questions with trustworthy evidence grounding and citation checking to reduce misinformation.
Authors
Sumit Barua, Guan Hong, Halil Dursunoglu, Charles Rodgers, Alvis Fong
Abstract
Retrieval-augmented generation (RAG) can improve access to complex information; however, retrieving evidence alone does not ensure that answers are grounded, citation-valid, or appropriately refused. This paper introduces CiteGuard-RAG, a validation-centered AI system for evidence-grounded question answering. The system integrates hybrid semantic-lexical retrieval, citation-constrained generation, sentence-level grounding validation, and single-pass regeneration. Validation is used at runtime to determine whether a candidate answer should be accepted, refused, or regenerated before final delivery. CiteGuard-RAG is evaluated on 400 questions across a controlled housing-law dataset, PrivacyQA, and CUAD. In the controlled evaluation, it achieves 99.1% retrieval accuracy, 98.3% grounded-answer accuracy, and 98.3% citation validity, with no validation-detected hallucinations. Ablation results show that grounded-answer accuracy drops sharply when validation is removed, even when retrieval accuracy remains unchanged. External evaluation shows that while citation validity remains strong, evidence utilization, span alignment, and refusal calibration become harder under domain shift. These findings indicate that trustworthy RAG systems require explicit validation between retrieval and final answer delivery. CiteGuard-RAG provides a practical architecture for linking retrieval, generation, citation checking, abstention, and regeneration in high-stakes information access.