Large language models improve search without adding false terms
Tracing and Taming Hallucinated Terms via Subsequence-Association Filtering and Breadth-Aware Selection for LLM Query Expansion
Information RetrievalComputation and Language
Summary
Large language models help improve search results by adding extra related words, but sometimes they make up incorrect or unrelated terms, which harms search quality. The authors propose a new way called SAFE-QE to spot and reduce these false words without needing extra training or changing the search system. Their method checks if suggested words reappear consistently in different contexts and whether the search results support them, keeping only reliable terms. Tests show it reduces incorrect additions while keeping search quality high.
What this means in practice
- •For search engine developers: Improve query expansion by reducing false added terms, enhancing search accuracy without retraining models or changing retrievers.
- •For advertising platform engineers: Maintain the relevance of expanded queries to reduce noise in ad targeting while using large language models for query understanding.
Authors
Amit Kumar Jaiswal, Sagar Uprety, Prayag Tiwari
Abstract
Large language models (LLMs) are now a standard source of query expansion terms but they hallucinate. They inject non existent entities and off-topic terms that add ranking noise and can hurt strong retrievers. Existing remedies such as corpus grounding, mutual verification and preference alignment treat hallucinated terms as an undifferentiated nuisance to be filtered by agreement heuristics, without a mechanistic account of why a given term appears or a principled criterion for how many terms to keep. We propose Subsequence-Association Filtered, breadth-aware query expansion (SAFE-QE), a training-free, retriever agnostic method that draws on the subsequence-association account of hallucination. We score each candidate expansion term by its corpus conditioned cross-context reproducibility, terms that reappear under randomised query contexts yet lack support in the pseudo-relevant set are flagged as products of dominant hallucinatory associations and down-weighted. Drawing on the characterisation of language generation as a trade-off between consistency (no hallucination) and breadth (coverage), we cast term selection as constrained coverage maximisation over the pseudo-relevant set, keeping a set that is approximately broad but consistent and stable across resampling. We also introduce the Hallucinated Expansion Term Rate, a query side faithfulness metric that counts injected expansion terms no relevant document supports. We evaluate our approach on benchmarked TREC and BEIR collections. Our SAFE-QE is statistically indistinguishable from generative and corpus-steered expansion on nDCG@10 while injecting fewer unsupported terms 4.3 per query against 27.8 for Query2doc and 57.5 for HyDE and on a strong learned sparse retriever, where every expansion method regresses. Faithfulness is therefore available at no measurable cost in ranking quality.