A Systematic Study of Pseudo-Relevance Feedback with LLMs
2026-03-11 • Information Retrieval
Information RetrievalComputation and Language
AI summaryⓘ
The authors studied how different ways of improving search queries using feedback impact performance. They focused on two aspects: where the feedback text comes from and how it is used to adjust the query. Their experiments show that the method of using feedback is very important, using text generated by language models is a cost-effective approach, and feedback from actual documents helps most when the initial search is strong. This helps clarify which parts of the feedback process matter most.
Pseudo-relevance feedbackLarge language modelsQuery representationFeedback sourceFeedback modelInformation retrievalBEIR benchmarkFirst-stage retrieverLow-resource tasks
Authors
Nour Jedidi, Jimmy Lin
Abstract
Pseudo-relevance feedback (PRF) methods built on large language models (LLMs) can be organized along two key design dimensions: the feedback source, which is where the feedback text is derived from and the feedback model, which is how the given feedback text is used to refine the query representation. However, the independent role that each dimension plays is unclear, as both are often entangled in empirical evaluations. In this paper, we address this gap by systematically studying how the choice of feedback source and feedback model impact PRF effectiveness through controlled experimentation. Across 13 low-resource BEIR tasks with five LLM PRF methods, our results show: (1) the choice of feedback model can play a critical role in PRF effectiveness; (2) feedback derived solely from LLM-generated text provides the most cost-effective solution; and (3) feedback derived from the corpus is most beneficial when utilizing candidate documents from a strong first-stage retriever. Together, our findings provide a better understanding of which elements in the PRF design space are most important.