Weakly adapting scientific embeddings improves literature mapping quality
Weakly Supervised Adaptation of Scientific Embeddings for Exploratory Literature Mapping
Digital LibrariesInformation Retrieval
Summary
Scientific papers can be represented as points in space using special mathematical tools called embeddings, which help find related work. However, these embeddings often become too similar when used on small, specialized collections of papers, making it hard to explore research themes clearly. The authors show that adapting these embeddings using publicly available citation data, without manual labeling, spreads out the points more evenly and forms clearer groupings. This method works better than just using basic word counts or citation links alone when organizing papers about artificial and machine consciousness.
What this means in practice
- •For digital library teams: Enhance navigation and clustering of small specialized research collections by adapting embeddings using existing citation metadata.
- •For academic search engine developers: Improve the quality of similarity measures for exploratory search in specialized research domains without requiring manual annotation.
Tested on one dataset.
Authors
Roman Neruda, Petra Vidnerová, Jiří Šejnoha, Jakub Mihálik, Tomáš Marvan
Abstract
Scientific document embeddings such as SPECTER2 are optimized for retrieval over large, heterogeneous corpora. On a small specialized corpus their similarity distributions can become highly compressed, which limits their usefulness for exploratory literature mapping. We study whether weakly supervised in-domain adaptation, using only metadata already available in open citation databases, changes this geometry. On ArtCon, a curated corpus of 704 publications on artificial and machine consciousness, we adapt the SPECTER2 proximity model with a lightweight LoRA adapter trained on positive pairs derived from bibliographic coupling in OpenAlex, using no manually annotated similarity labels. We compare the adapted embeddings with lexical (TF-IDF), citation-based (bibliographic coupling) and generic semantic (SPECTER2) representations under thresholded, fixed-density and symmetric k-nearest-neighbour graph constructions. Adaptation substantially decompresses the similarity distribution (mean pairwise cosine similarity 0.9104 to 0.8663, standard deviation 0.0259 to 0.0594) and yields graphs with markedly higher Louvain modularity and better connectivity than the base encoder under matched edge and neighbourhood budgets. The effect is stable across all three graph constructions and is therefore not an artefact of a single sparsification choice. We report this as a case study on one corpus, and we are explicit about what it does not establish: the clustering indices used here are computed against community labels derived from each representation's own similarity graph, so they measure internal geometric self-consistency rather than agreement with an external topical ground truth, and adaptation and evaluation use the same documents. Evaluation against independent labels, on held-out documents, and against geometric controls is ongoing work.