Measuring citation accuracy tradeoffs in compressed retrieval generation
The Attribution-Compression Frontier in Retrieval-Augmented Generation
Computation and LanguageArtificial Intelligence
Summary
When computers generate answers using retrieved text, they often need to shorten or compress this text to fit into memory or speed up processing. This paper studies how different ways of shortening the text affect not only the quality of answers but also how accurately the computer can cite the sources of the information. The authors find that some compression methods keep answer quality but lose citation accuracy, meaning the system is less trustworthy about where information came from. They also point out that evaluation depends a lot on the tools used and that more human verification is needed.
What this means in practice
- •For chatbot developers: Improve source citation reliability in compressed input used by chatbots to keep answers trustworthy while handling limited context size.
- •For legal document teams: Help legal tech systems verify and cite supporting text accurately when summarizing or compressing large sets of legal documents.
Authors
Deepanshu Mody
Abstract
Context compression reduces generator input in retrieval-augmented generation, but answer quality alone does not characterize citation attribution. We measure citation attribution across compression methods and budgets, comparing reranking, extractive selection, abstractive summarization, token pruning, and an extract-cluster-rewrite construction on ASQA and QASPER under a fixed generator and primary entailment evaluator. On ASQA at a nominal 0.25 budget (achieved compression 0.08), a RECOMP-style compressor's citations score 0.86 precision against its summaries but 0.12 against source spans under our re-attributability protocol. These estimates depend on a shared NLI model for span recovery and citation scoring and lack independent human calibration. Extractive selection's observed grounded precision ranges from 0.43 to 0.49 across nominal budgets from one-half to one-tenth of the ASQA context, while answer quality declines. For the same RECOMP setting, claim verification after source recovery yields an unsupported rate of 0.88 versus 0.17 when checking summaries. This gap persists beyond structural rejection of missing provenance, but remains evaluator-dependent. A 200-question TRUE T5-XXL audit also finds emitted--grounded gaps under both fixed and recomputed source mappings, without establishing human-calibrated support rates.