DeBERTa ensembles verify citation truthfulness in scientific reports
NLPCC 2026 Task 10: Citation-Level Faithfulness Verification with DeBERTa Ensembles and Class-Wise Calibration
Computation and Language
Summary
Scientific reports often use AI to help write claims that refer to other research papers. This paper presents a system that checks if a specific scientific claim is true according to the cited paper and also points out exactly which paragraphs support it. The system combines multiple AI models and techniques to improve accuracy without needing external searches or large language model prompts. It performed well in a public competition, ranking second with strong overall results.
What this means in practice
- •For scientific publishing teams: Automatically verify the truthfulness of scientific claims against their source documents to improve report reliability.
- •For legal compliance teams: Detect and verify citation accuracy in technical documents to ensure factual correctness in regulated environments.
Authors
Yanling Li, Zirui Li, Mingyu Wan
Abstract
This paper presents our system for Track 2 of the NLPCC 2026 Shared Task 10 on citation-level faithfulness in AI-assisted scientific reporting. Given an atomic scientific claim and the structured full text of its cited paper, the task requires both a four-way relation label and up to three evidence paragraph identifiers. The label head ensembles a paragraph-aware cross-encoder with a document-level DeBERTa-large classifier, followed by class-wise decision calibration. Probability-level fusion is motivated by an out-of-fold tendency to over-predict Topical Match. The evidence head combines paragraph scores from top-20 and top-30 joint models with BM25 scores. The system runs fully offline without external retrieval or LLM prompting. On the final leaderboard, our system achieved 82.9898 overall (89.5491 Macro-F1 and 76.4305 Joint@3), ranking second in Track 2. Ablations and error analysis show that model complementarity and calibration drive the label gains. Gold-evidence inference changes label Macro-F1 negligibly, whereas evidence ranking remains important for Joint@3.