LLM helps measure true impact of research beyond citations
From Citations to Contributions: LLM-Assisted Credit Scoring of Research Articles
Digital LibrariesArtificial IntelligenceComputation and LanguageInformation Retrieval
Summary
Counting how many times a research paper is cited doesn't always show how much it actually contributed to science. The authors created a way to figure out what part of a paper is original and what part builds on earlier work. They use large language models to estimate how important each part of the paper is, then spread that credit through connected papers to understand influence better. This method helps see the real contributions papers make, not just how often they are mentioned.
What this means in practice
- •For research library managers: Rank and recommend scientific articles by their original contributions rather than citation counts alone to improve collection usefulness.
- •For academic publishing teams: Provide more nuanced metrics of article impact that reflect unique contributions to improve editorial decisions and author recognition.
Authors
Sana Ebrahimi, Suraj Shetiya, Abolfazl Asudeh
Abstract
Citation-based measures of scientific influence typically treat citations as uniform signals, ignoring the different roles that cited works play in a paper's contribution. We introduce contribution-based credit scoring for research articles: a structured citation analysis that decomposes a paper's credit between its own original contribution and the prior work it builds on. Motivated by a cooperative-game view of scientific credit, we propose the contribution tree, a hierarchical framework that conserves importance across the document structure and separates original from citation-derived contribution. To make this framework scalable, we use LLMs as noisy comparative estimators of local importance. We further extend the model to article collections by propagating contributions through weighted citation graphs, yielding corpus-level contributions and normalized influence scores. Our experiments suggest that our framework captures contribution signals beyond surface-level heuristics. Our code is available at https://github.com/sanaebrahimi/Importance_Scoring/