Summary
Finding information from large document collections can be hard because many systems try to represent each document with just one label or identifier, which can miss important details. The authors propose a new method called Passage-Driven Multi-ID Retrieval (PDMR) that breaks documents into smaller parts and assigns each part its own identifier. This way, search queries can match specific parts of a document instead of relying on a single overall label, improving accuracy. The researchers tested PDMR on popular datasets and found it performed better than existing methods, especially when identifying the most relevant documents. They also showed that several components of their approach each help make the system work well.
Generative retrievalDocument identifiersPassage segmentationMulti-target learningRecallMRR (Mean Reciprocal Rank)NQ320KMS MARCOAutoregressive models
Abstract
Generative Retrieval (GR) models map queries directly to document identifiers, replacing conventional retrieval over external sparse or dense indexes with autoregressive identifier generation. However, most generative retrieval frameworks rely on a single-identifier assumption, mapping each document to a single target sequence. This forces the model to represent all document content with one sequence. Since documents are often multi-faceted, this can lead to lossy representations and reduced robustness to query variation, where multiple query intents must compete for a single generative access path. In this work, we introduce Passage-Driven Multi-ID Retrieval (PDMR), a generative retrieval framework that represents documents through multiple passage-level identifiers. PDMR segments each document and assigns one identifier to each selected passage, which provides multiple semantic entry points for retrieving the same document. This multi-entry representation allows the model to align queries with specific semantic facets, thereby reducing the dependence on a single document-level target. To address the supervision ambiguity of this one-to-many mapping, we formulate training as a multi-target learning problem and explore an objective function designed to distribute probability mass across multiple valid passage-level identifiers. We evaluate PDMR on NQ320K and MS MARCO Document. On NQ320K, PDMR improves over strong generative and non-generative baselines on Recall@1 and MRR@100. On MS MARCO Document, PDMR achieves the best Recall@1 and MRR@10 among the reported methods, while remaining competitive on Recall@10. Controlled ablations further show that passage-level supervision, identifier design, training-query augmentation, and multi-target learning contribute complementary gains.