Scientific Claim-Source Retrieval Revisited: A Comparative Study of Style Transfer and Re-Ranking
Information Retrieval
Summary
The authors studied how to automatically find the original scientific paper behind a claim shared on social media, which is hard because claims are often written differently than the source. They tested different ways to retrieve these source documents, showing that translating claims into English works better than using the original language, and adding extra information about the paper helps too. They also tried changing the style of the claims to better match sources, which improved results depending on the method used. Finally, the authors developed new ways to re-rank search results, with their verification-based method performing the best overall.
Authors
Tobias Schreieder, Harsh Khandelwal, Yu-Ling Zhong, Michael Färber
Abstract
Scientific claims shared on social media are often difficult to verify and may contribute to the spread of misinformation. To address this challenge, automated fact verification systems require scientific claim-source retrieval, the task of identifying the source publication underlying a given claim. However, claims often differ substantially from their source publications in language, style, and specificity, making retrieval challenging. We present a comparative study of scientific claim-source retrieval on the CheckThat! 2026 benchmark across sparse and dense retrieval models. Our results show that translating claims into English outperforms both original and bilingual claim representations, while incorporating publication metadata provides additional retrieval gains by capturing indirect source references. In addition, we analyze four style transfer approaches and find that they improve retrieval performance for most models, although the optimal style depends on the underlying retrieval objective. Finally, we investigate similarity- and signal-based re-ranking approaches, introducing three novel re-ranking models based on attribution, entity overlap, and verification-based reasoning. Verification-based re-ranking yields additional gains beyond semantic similarity and achieves the best overall performance with an MRR@5 of 0.758.