Semantically Similar, Logically Distinct: Diagnosing the Semantic-Answerability Gap in Table RAG
2026-07-20 • Artificial Intelligence
Artificial Intelligence
AI summaryⓘ
The authors study how well models can find the right table to answer a question when many similar tables exist. They create a new test called TCR-Bench and find that current retrieval models often pick tables that are related but not the one that actually answers the question. This is called the Semantic-Answerability Gap and happens because the models rely too much on overall similarity instead of confirming if the table truly contains the needed answer. They propose a two-step method that checks answerability first, which greatly improves the chances of retrieving the correct table. This shows that verifying answerability is important and can fix much of the existing problems.
retrieval-augmented generationtablesanswerabilitydense retrieverssemantic relevanceTCR-BenchSemantic-Answerability Gapquery-table matchingrerankinginformation retrieval
Authors
Jiaming Tian, Liyao Li, Wentao Ye, Haobo Wang, Lihua Yu, Zujie Ren, Gang Chen, Junbo Zhao
Abstract
Tables are a critical knowledge source in retrieval-augmented generation (RAG), but a retrieved table may lack sufficient evidence to answer a query, a property we call answerability. While answerability broadly concerns whether a source or collection of sources contains sufficient evidence, retrieval models optimized for semantic relevance do not guarantee it even in the single-source case, creating a fundamental mismatch. To study this, we introduce TCR-Bench, a diagnostic benchmark for Table Content-level Answerability in RAG, built around sibling tables, i.e., tables with highly similar schemas but subtle content differences. On TCR-Bench, the dense retrievers we evaluate persistently exhibit a Semantic-Answerability Gap: they often retrieve the correct sibling group yet struggle to pinpoint the uniquely answerable table within it, dropping QA performance from 0.755 (oracle) to 0.330 (top-5 retrieved). Our analysis suggests this gap is associated with semantic accumulation, schema-level cue dependence, and weak row-column binding. As a diagnostic probe into the source of this gap, we test whether a lightweight two-stage pipeline, Answerability-Aware Reranking (AAR), applying direct query-table answerability judgment, can recover performance: it raises top-1 target retrieval from 18.2% to 57.4%, and this large gain is itself evidence that much of the observed failure reflects a missing answerability verification step, rather than an inherent limitation of model capacity alone.