Papers for

research management teams

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Scientists detect shifts in research beliefs years before citations do

Citations Are Late: Reading epistemic instability from what papers believe, years before the citation graph catches up

Abstract: Paradigm shifts in science are visible in what researchers assert and contest before they are visible in the citation graph. We ask whether a cheap, content-level signal of epistemic instability, derived from the changing distribution of stated modelling beliefs in paper abstracts, can anticipate a paradigm shift earlier than the dominant citation-based disruption index (CD5). On the displacement of recurrent networks by Transformers in NLP, a pre-registered content signal crosses its detection threshold in 2016-Q1, whereas a real-time CD5 monitor cannot even observe the 2017 breakthrough until 2022-Q2, since CD5 needs a five-year forward-citation window: a lead of about 25 quarters. The flat citation baseline is not an artifact of one index, as our CD5, a reference-normalised variant, and an authoritative precomputed index all sit near zero across the shift. The lead is also not a faster proxy: at equal latency the signal beats the content competitors tested, including a learned CD-from-text model and an embedding disruption measure, and the decomposition sees what a keyword cannot (keyword-blind AUC of about 0.9, reproduced on human labels). The lead over CD5 is an observability lead. Which signal carries it depends on the shift: belief adoption in NLP, contestation and applicability stress in vision. Measured against the breakthroughs themselves, the signal leads by five quarters in NLP and is contemporaneous in computer vision.

Mon 5 OctDigital Libraries
The gist
Scientific breakthroughs often show up in what researchers say before citation counts reflect them. The authors show that analyzing the changing beliefs expressed in paper summaries can reveal big shifts in fields like natural language processing years earlier than traditional citation-based methods. They tested this on the rise of Transformer models replacing recurrent networks in language tasks and saw clear signals about 6 years before citations caught up. This approach can help spot important changes as they happen, rather than after a long waiting period.
Open → 2610.07397v1