Embedding geometry reveals shifts in scientific ideas over time
Geometric Signatures of Conceptual Reorganization: A Counterfactual Embedding Framework for Detecting Scientific Revolutions
Digital LibrariesComputation and LanguageMachine Learning
Summary
Some big changes in science happen when new ideas reorganize what scientists think and know. The authors show how looking at the math shapes that represent scientific papers can detect these changes. They do this by seeing what happens when they remove papers about an idea before and after it became important. They tested their method on famous breakthroughs in physics, math, and AI, finding clear geometric signs of these shifts. This gives a new way to study how science changes and grows.
What this means in practice
- •For science historians: Identify quantitative evidence of major conceptual shifts in scientific fields from historical publication data.
- •For knowledge management teams: Detect reorganizations of research topics within large scientific literature databases to inform strategic planning.
Authors
Dimitris Ntounis, Ariel Schwartzman, Chris Chafe, Thomas A. Ryckman
Abstract
We introduce document embedding geometry as a quantitative observable of conceptual reorganization and develop a counterfactual ablation framework for measuring how individual concepts influence the organization of scientific knowledge, providing a quantitative framework for detecting scientific revolutions. The observable is defined by the geometric perturbation induced when removing documents associated with a candidate concept from the embedding space before and after its historical emergence. Statistical validation is performed using five historical case studies spanning physics, mathematics, and machine learning: special relativity, Gödel's incompleteness theorems, the Higgs mechanism, deep learning, and the attention mechanism underlying transformer architectures. Across the historical case studies, the framework identifies measurable geometric signatures associated with conceptual reorganization, while the validation studies expose important limitations arising from document assignment and sparse historical data. These results establish embedding geometry as a medium for quantifying conceptual reorganization, providing a new approach for studying how scientific fields restructure over time.