Generative retrieval improves documents by using multiple passage identifiers
PDMR: Passage-Driven Multi-ID Document Retrieval
Information Retrieval
Summary
Finding information in long documents is hard because a single reference doesn’t capture all the details. The authors created a new way to find documents by breaking them into smaller sections, each with its own identifier. This lets the method match specific parts of a document to different search queries more accurately. They trained their system to handle multiple correct answers and tested it on standard datasets, where it performed better than existing methods.
What this means in practice
- •For search engine developers: Improve document search by enabling retrieval systems to match queries to multiple detailed parts of a document instead of relying on a single identifier.
- •For enterprise knowledge management teams: Enhance internal document search accuracy by indexing documents through passage-level identifiers that better capture varied user intents.
Authors
Smail Oussaidene, Mohand Boughanem
Abstract
Generative Retrieval (GR) models map queries directly to document identifiers, replacing conventional retrieval over external sparse or dense indexes with autoregressive identifier generation. However, most generative retrieval frameworks rely on a single-identifier assumption, mapping each document to a single target sequence. This forces the model to represent all document content with one sequence. Since documents are often multi-faceted, this can lead to lossy representations and reduced robustness to query variation, where multiple query intents must compete for a single generative access path. In this work, we introduce Passage-Driven Multi-ID Retrieval (PDMR), a generative retrieval framework that represents documents through multiple passage-level identifiers. PDMR segments each document and assigns one identifier to each selected passage, which provides multiple semantic entry points for retrieving the same document. This multi-entry representation allows the model to align queries with specific semantic facets, thereby reducing the dependence on a single document-level target. To address the supervision ambiguity of this one-to-many mapping, we formulate training as a multi-target learning problem and explore an objective function designed to distribute probability mass across multiple valid passage-level identifiers. We evaluate PDMR on NQ320K and MS MARCO Document. On NQ320K, PDMR improves over strong generative and non-generative baselines on Recall@1 and MRR@100. On MS MARCO Document, PDMR achieves the best Recall@1 and MRR@10 among the reported methods, while remaining competitive on Recall@10. Controlled ablations further show that passage-level supervision, identifier design, training-query augmentation, and multi-target learning contribute complementary gains.