Scientific entity mentions improve claim extraction from research papers
Towards Full-Text Scholarly Argument Mining using Entity-Anchored Claim Extraction
Computation and LanguageInformation Retrieval
Summary
Finding arguments in full research papers is hard because the text is long and complicated. The authors use mentions of scientific things like tasks and models to find claims more easily. They checked their method on 50 sentiment analysis papers and created a reliable set of claims using human agreement. Then they used large language models to make a bigger set of claims, which helped train better computer models for this task. These models also worked well when applied to papers outside of the initial set.
What this means in practice
- •For software developers: Automatically identify and structure claims in scientific articles to support building literature analysis tools.
- •For academic publishers: Enhance document indexing by extracting claims anchored to scientific entities for better search and summarization services.
Tested on one dataset.
Authors
Bhuvanesh Verma, Andy Lüking, Mounika Marreddy, Leon Lukas Hammerla, Alexander Mehler
Abstract
Argument mining in full-text scholarly articles remains an important yet challenging task. This difficulty stems not only from the need to model long-distance dependencies but also from the substantial effort required to annotate sentence-level argumentative components throughout lengthy documents. To address these challenges, we introduce a novel approach that leverages scientific entity mentions (e.g., tasks, datasets, and models) to identify and structure claims in scholarly literature. We validate our approach through a case study involving 50 full-text research papers on sentiment analysis. We first establish a reliable human gold standard (Krippendorff's $α= 0.80$), identifying 113 claim-bearing sentences and extracting 199 structured claim records from 300 candidate sentences. To address the annotation bottleneck, we employ a majority-voting ensemble of large language models (LLMs) to generate a silver-standard dataset comprising 1,487 claims. Encoders fine-tuned exclusively on the silver-standard dataset outperform those trained on a small human-annotated gold-standard dataset and demonstrate strong zero-shot transfer to NLP papers outside the training corpus.