Longstanding problems in measuring research impact by country
On some unresolved issues in aggregate level comparative research evaluation
Digital Libraries
Summary
Measuring and comparing research impact between countries is complicated by some old, unresolved problems. One issue is that global reference scores depend on which research units are included and their scores, which can distort comparisons. Another challenge is that international collaboration makes citation impacts converge, reducing the accuracy of country-level impact measurements. The authors highlight these problems and suggest new ways to address them, discussing the pros and cons of past and potential solutions.
What this means in practice
- •For research funding agencies: Improve assessment methods to better differentiate country research impact when allocating resources internationally.
- •For science policy analysts: Develop more accurate evaluation frameworks that account for collaboration effects on country citation impact statistics.
A position paper. It proposes an approach and reports no results.
Authors
Paul Donner
Abstract
This short conceptual paper recalls to the attention of the scientometrics community some long-standing open methodological problems and proposes new directions that might be taken in renewed engagements with them. These problems are the differential dependency of the global reference score on the included units' observed scores and the dilution of impact measurement accuracy due to collaboration, which causes a global convergence in country citation impact due to international collaborative research. Advantages and disadvantages of existing and new solutions are discussed.