Comparing ways language models find documents by decoding and IDs
Disentangling Paradigm, Identifier, and Decoding in Generative Retrieval
Information RetrievalComputation and Language
Summary
Finding the best document for a question can be done by having a language model generate a document's ID. The authors studied different decoding methods and types of IDs used in these models, showing that how the ID is decoded significantly affects accuracy. Their experiments found that models mostly memorize which ID matches which question rather than understanding the content deeply. They also show that comparing different model types requires careful matching of how they are trained and decoded.
What this means in practice
- •For search engine engineers: Improve document retrieval efficiency by selecting the best combination of decoding methods and identifier types evaluated in this paper.
- •For machine learning deployers: Choose appropriate generative retrieval models and decoding strategies based on performance trade-offs detailed in this evaluation.
Authors
Hicham Randrianarivo, Logan Renaud, Alexia Allal
Abstract
Generative retrieval trains a language model to generate the identifier of a relevant document. Recent work replaces the autoregressive decoder with diffusion, but changes identifiers, training recipe and decoding at once, so differences cannot be credited to the paradigm. On NQ320K and MS300K, we train autoregressive, masked-diffusion and block-diffusion models with residual-quantised, product-quantised and random identifiers. With identifier length and training budget fixed, we decode each model in several ways. Decoding alone moves a diffusion model's Hit@1 by 6.6 to 13.7 points. Our reference diffusion decoding, generate-and-match, generates an identifier, then retrieves the closest corpus identifiers. The generated identifier is right for 14-21% of NQ320K queries. We test one-pass scoring to decode diffusion retrievers: the model reads a fully masked identifier once, and each document is scored by its codes' probabilities. It matches or beats generate-and-match in 11 of 12 settings. Autoregressive models still lead in Hit@1; on NQ320K, the lead comes from the model, not beam search. Starting from one sampled identifier, one-pass scoring removes 46-83% of masked diffusion's deficit to beam search; from generate-and-match, at most a quarter. On NQ320K, every paradigm largely memorises which identifier answers which query: random identifiers keep 83-90% of the Hit@1 of residual-quantised ones. There, product-quantised identifiers lead residual-quantised ones by 3.4 points in the autoregressive model and by -0.7 to +3.6 in diffusion models; across decodings, AR's gap exceeds diffusion's by 1.5-2.3 points, around our 2-point threshold. Paradigm comparisons must report each paradigm at its own recipe and best decoding.