The Matryoshka Hypencoder
2026-07-20 • Information Retrieval
Information Retrieval
AI summaryⓘ
The authors explain an improved version of Hypencoder, a method that turns search queries into small neural networks to find relevant documents. They extended it to use different sized networks, letting systems choose between speed and accuracy. Their new design, called the Matryoshka Hypencoder, keeps accuracy close to the original but uses fewer active network parts, making it faster when searching. This approach could help make these systems easier to use in real-world settings.
Hypencoderquery encodingneural networksdocument embeddingsMatryoshka Representation Learningmodel efficiencyretrieval systemsscoring throughputparameter trade-off
Authors
Majd Alkawaas, Sean MacAvaney
Abstract
The Hypencoder is a recently-proposed retrieval approach that encodes queries as shallow neural networks ("Q-Nets") that estimate relevance over pre-computed document embeddings. Inspired by Matryoshka Representation Learning, we show that the Hypencoder can be extended to support multiple sizes of Q-Nets, allowing trade-offs between effectiveness and efficiency when deployed. We find that this "Matryoshka Hypencoder" achieves comparable in-domain effectiveness with approximately 7x fewer active parameters in-domain and half as many active parameters out-of-domain, which corresponds to a 1.6-3.4x increase in scoring throughput. This work paves the way for practical deployment of Hypencoders.