Scientific retrievers improve rare concept understanding with verification
REPAIR: Resolving Long-Tail Confusion in Scientific Retrievers via Fact-Verified Iterative Refinement
Artificial Intelligence
Summary
Scientific search tools often struggle to find precise information about less common or specialized topics because they lack enough good examples to learn from. The authors present REPAIR, a new method that improves these tools by creating better training data through a cycle of spotting difficult topics, expanding evidence using APIs, and carefully selecting challenging examples. This method helps the search tools make more accurate distinctions and reduce errors. Tests show that REPAIR works better than many existing methods in fields like materials science and biomedicine.
What this means in practice
- •For biomedical data teams: Enhance biomedical information retrieval accuracy by improving handling of rare and complex scientific terms in search queries.
- •For materials science software developers: Create more precise materials science knowledge retrieval systems with fact-verified iterative refinement to reduce retrieval errors.
Authors
Yerim Oh, Gunhee Kim
Abstract
Precise retrieval of scientific information is fundamentally constrained by long-tailed concepts and high fact-sensitivity of scientific corpora. These challenges often limit the effectiveness of dense retrievers and hallucination-prone LLM augmentation. To address this, we present REPAIR, a self-evolving data augmentation framework for scientific dense retrievers. REPAIR iteratively synthesizes training data to address knowledge gaps by cycling through diagnosis of long-tail concepts, API-guided evidence expansion, and differentiation via hard negative mining. This process effectively grounds retrieval in factual reality to resolve fine-grained distinctions. Extensive experiments demonstrate that REPAIR significantly outperforms 19 strong baselines on nine materials science and biomedical benchmarks. Our work highlights that diagnosing and factually augmenting data to long-tail deficits is essential for robust scientific retrieval.