Hybrid retrieval fusion can lower quality on some search queries

The Fusion Tax: A Diagnostic Analysis of When Hybrid Retrieval Hurts

Information RetrievalComputation and Language

Summary

When combining two ways to find documents—one based on keywords and another using neural networks—the combined approach doesn’t always work better for every search. The authors studied when mixing these methods actually makes search results worse for individual queries, despite helping on average. They found that this problem, which they call a fusion tax, happens most when only one method finds relevant documents well. Their work explains when to avoid mixing these methods and how to predict when fusion might hurt search quality.

What this means in practice

  • For search engine developers: Decide when to combine lexical and neural search methods to maintain or improve individual query result quality based on dataset balance.
  • For enterprise search teams: Improve internal document search by tailoring sparseneural fusion based on measured balance, reducing cases where fusion worsens results on important queries.

Authors

Aaditya Chauhan, Arun Agarwal

Abstract

Hybrid retrieval, which fuses a sparse lexical retriever with a dense neural one, is widely recommended in the software documentation surveyed in September 2026, usually with reciprocal rank fusion (RRF), on the strength of aggregate gains. We ask what fusion costs on individual queries, treating "do not fuse" as a first-class outcome, in the setting in which hybrid retrieval is deployed: a standard Anserini BM25, five dense encoders spanning 22M-596M parameters and three model generations, a learned-sparse (SPLADE) alternative, and a cross-encoder second stage, on seven BEIR datasets and three fusion functions. Even the best method, dense-weighted score interpolation, ranks relevant documents below the better of its two components on 13-48% of queries and below the deployable dense-only retriever on 8-21% (14-48% and 11-25% at a weight selected under leave-one-corpus-out validation), while beating dense-only in aggregate on every corpus with the primary encoder; the downside is significant under loss-averse risk-sensitive evaluation on six of the seven corpora. We call this loss the fusion tax. It is mostly a ranking effect that preserves recall in aggregate, it is most likely on the queries that only one retriever covers, and it persists across the encoder ladder and the SPLADE substitution. RRF carries the higher tax wherever the dense retriever is clearly stronger and only beats dense-only where the two components are balanced. Fusion failure is weakly predictable within a corpus but transfers unreliably across corpora, because the features that predict it change sign with the sparse-dense balance. A cross-encoder recovers part of the tax only where it is itself reliable. We distill these findings into guidance that keys the fusion decision to the measured sparse-dense balance of the target corpus.