Trace system improves precise answers from scientific papers with table extraction
TRACE: Target-Aware Retrieval, Attributed Evidence, and Contract-Constrained Extraction for LitTraceQA
Computation and LanguageDigital LibrariesInformation Retrieval
Summary
Finding the right scientific paper is different from getting exact answers from it. The authors address this by making a system called TRACE that finds the right paper, locates evidence precisely, and extracts answers in the right format. TRACE works well by organizing information from many papers, understanding tables visually and structurally, and validating answers carefully. It performs well on a test set, especially on identifying the paper and choosing the right answer type, though it still struggles with details like exact locations in tables.
What this means in practice
- •For technical document teams: Produce verified precise answers from scientific papers for compliance or review using targeted retrieval and exact evidence extraction methods.
- •For business intelligence analysts: Improve automated extraction of structured data from scientific tables to support data-driven decision making in technical domains.
Authors
Sachin Gupta, Divya Godara
Abstract
Finding a relevant paper is not the same as producing a verifiable answer from it. LitTraceQA requires canonical paper identifiers, exact evidence at the page or object level, and typed answers that match the evaluator. We call the separation between source access and scorer-visible correctness the grounding contract gap. TRACE - Target-Aware Retrieval, Attributed Evidence, and Contract-Constrained Extraction - addresses this gap with target-grouped retrieval, independent typed evidence localization, multimodal table extraction, schema-driven table construction, and fail-closed validation. It indexes 27,487 papers through passage, object, alias, citation, and dense representations while retaining the question target behind each signal. For tables, TRACE predicts the observation unit before extracting values and assembles rows with evaluator-compatible key normalization. Our audited selected clean-track artifact scores 0.760613 on the official 71-question test set, including 0.9728 paper F1, 0.6847 evidence F1, 0.9800 multiple-choice accuracy, 0.5423 table-row F1, and 0.3508 macro cell accuracy. On 11 public-development table records, a clean baseline and coordinate-aware visual fill obtain row F1 of 0.291 and 0.411, respectively; this diagnostic comparison includes fallback outputs and is not an official-test claim. Remaining errors chiefly concern locator, observation-unit, row-key, and source-value identity.