Papers for

academic search platform developers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Science maps show consistent patterns but vary on close paper details

How much does the map of science depend on the embedding model?

Abstract: Science maps describe relationships between papers. Text embedding models build these relationships by turning a paper's text into a list of numbers that gives it a position. Would another model give the same picture? I compare ten models using titles and abstracts of 500,000 publications from the OpenAlex bibliographic catalogue, published in 2000-2024 across 26 disciplines, including Medicine and Physics. Representing each discipline by its papers' average position produces similar arrangements across models, including when other papers are selected. Yet within a discipline and period, two models share, on average, only eight of the 25 papers closest to a given paper. Supplying titles alone also changes much of that list, with effects that vary between models. Models can also disagree about which discipline's papers are more spread out. One measure of spread compares angles between lines drawn from a common point to each paper's position. Moving that point can reverse which discipline appears more spread out, even when selecting other papers leaves the answer unchanged. The broad arrangement can remain similar while answers about nearby papers and differences between disciplines depend on the model, the text it receives and how its output is processed.

Mon 12 OctDigital Libraries
The gist
Science maps use math models to show how research papers relate based on their text. The authors compared ten different models to see if they produce the same map. They found that while big-picture views of scientific fields look similar across models, the details about which papers are closest to each other can vary a lot. What text is used and how the map is analyzed also change the results. So, the overall map is somewhat stable, but the specifics depend on the model and input.
Open → 2610.16639v1