Large Language Models for Citation Function Classification
2026-07-20 • Computation and Language
Computation and Language
AI summaryⓘ
The authors studied how well different advanced language models can classify the purpose of citations in scientific papers, such as whether a citation praises, criticizes, or neutrally mentions another work. They tested five models using various training methods and found that fine-tuning the Falcon 7B model gave the best results on a popular dataset (ACL-ARC). They also created a new dataset called AC3 with more detailed citation categories and tested how the amount of context affects classification. Their work offers a useful comparison of models and insights for future research in this area.
citation function classificationlarge language modelsfine-tuningzero-shot learningfew-shot learningACL-ARC datasetAC3 datasetcontext extractionmacro F1 scorebibliometric analysis
Authors
Daniel Vodička, Jakub Šmíd, Pavel Král, Christophe Cerisara
Abstract
Citation function classification plays a crucial role in understanding the relationships between scientific publications and advancing bibliometric analysis. This study presents one of the first comprehensive evaluations of multiple state-of-the-art (SOTA) large language models (LLMs) for citation function classification, achieving new SOTA results on the ACL-ARC dataset. We systematically compare five models (Mistral 7B, Orca 2-7B, LLaMA 3.1-8B, Falcon 7B, and SciBERT) across zero-shot, few-shot, and fine-tuning approaches. Our fine-tuned Falcon 7B model achieves a 73.3% macro F1 score on ACL-ARC, representing a significant improvement over previous methods. Additionally, we introduce AC3, a novel dataset featuring a seven-category annotation scheme that distinguishes between neutral acknowledgments and explicit evaluative stances (more opinion-oriented citations - criticizing, complimenting, contradicting). The dataset is implemented across four context extraction variants to systematically evaluate the impact of contextual scope on classification performance. We also provide detailed analysis of model performance, experimental configurations, and limitations to guide future research in this domain. To our knowledge, this is one of the first studies dedicated to comprehensive model comparison for citation function classification, addressing a gap identified in recent surveys.