RefVerifier helps verify scientific paper claims faster and more accurately
RefVerifier: Semi-Automated Reference Claim Verification for Scientific Manuscripts
Software Engineering
Summary
Scientific papers often cite other studies to support their claims, but checking these references carefully takes a lot of time for reviewers. The authors created RefVerifier, a tool that semi-automates this process by finding citation sentences, verifying bibliography details, and locating evidence in open-access papers. It then gives clear explanations helping reviewers decide if claims are supported. Their tests show RefVerifier is very good at spotting claims and finding evidence passages, making peer review more efficient without losing human judgment.
peer reviewcitation verificationbibliography metadataopen-access PDFevidence localizationclaim detectionfact checkingscientific manuscriptsreference resolution
Authors
Stefania Mocan, Florian Angermeir, Mark Kreitz
Abstract
As software engineering research submission counts surge, peer reviewers face severe time constraints, making systematic verification of citation-supported claims prohibitively expensive. Consequently, unsubstantiated claims and semantic drift can propagate undetected across scientific literature. Existing approaches such as fact-checking and retrieval-augmented generation tools operate on open-domain web data or evaluate claims in isolation without processing complete manuscripts. To address this gap, we present RefVerifier, a semi-automated, citation-bounded reference verification prototype designed to support in academic peer review. RefVerifier extracts citation-bearing sentences from manuscripts, checks bibliography metadata against scholarly databases, resolves references to full-text open-access PDFs, localizes relevant evidence passages, and generates verdicts with natural language explanations. Evaluating RefVerifier on public benchmarks shows claim detection at an F1 score of 0.990, open-access resolution of 57.6% of references, and evidence localization with a hit rate of 98% on abstracts and 68% on complete cited papers. In an end-to-end test with eight manuscripts, RefVerifier achieves a verdict accuracy of 71%. By automating document retrieval and evidence localization while preserving reviewer oversight, RefVerifier provides first indicators for the feasibility of semi-automated integrity checks in scholarly publishing.