Papers for

digital library teams

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Compact multilingual retrievers match larger models for European languages

Parameter-Efficient Retrievers for Polish and European Languages

Abstract: Dense retrieval systems increasingly rely on multi-billion-parameter language models, whose memory and computational requirements make large-scale indexing, frequent corpus updates, and low-latency serving costly. We present a three-stage training pipeline for developing compact and efficient retrievers that remain competitive with substantially larger models. The pipeline combines cross-lingual alignment, relational knowledge distillation, and contrastive fine-tuning. It requires no original ground-truth relevance labels, relying exclusively on supervision generated by strong embedding models and rerankers utilised as teachers. Using this pipeline, we develop PolDense and EuroDense, both supporting contexts of up to 8,192 tokens. PolDense is a family of six Polish retrievers ranging from 17M to 1B parameters. EuroDense is a 435M-parameter retriever supporting nine European languages. We conduct an extensive evaluation covering 41 Polish and 150 multilingual retrieval tasks. The results demonstrate strong quality-efficiency trade-offs. PolDense-1B outperforms the evaluated retrievers with up to 9B parameters, while the PolDense family forms the Pareto frontier across model sizes. Among the evaluated models below 1B parameters, EuroDense ranks first in both task-averaged and language-averaged performance and leads in seven of nine languages. We release all models publicly.

Fri 11 SeptComputation and Language
The gist
Large language models help computers find relevant documents but are very big and slow, making them hard to use frequently or for many languages. This paper shows a new way to train smaller, faster search tools that work almost as well as the biggest ones, especially for Polish and other European languages. The authors used clever training techniques that don’t need lots of labeled examples but learn from existing strong models. They made several versions, some very small and some larger, and tested them widely, finding these compact models often do better than much bigger ones.
Open 2609.12913v1