Embedding reranker training improves legal document search quality
Beyond Recall Saturation: Scaling Hard-Negative Reranker Training for Industrial Legal Retrieval
Computation and Language
Summary
Finding the most relevant legal documents when searching can be tricky, especially if the system only looks at a small set of candidates. The authors show that expanding the list of candidates to about 300 already finds most relevant documents, but ranking those correctly is still a challenge. They tested different ways to train the system that sorts (reranks) these documents and found a two-step approach works best. Their improved model not only scores better on legal benchmarks but is also preferred in tests using real search queries, helping legal search systems be more accurate without slowing down too much.
What this means in practice
- •For legal tech teams: Enhance legal document search by improving reranker training using a staged hard-negative mining approach to boost relevance without excessive slowdown.
- •For enterprise search engineers: Improve large-scale text retrieval systems by separately optimizing embedding retrieval and reranking phases, leading to better overall search quality.
Authors
Yunhan Li, Mingjie Xie, Zeyang Shi, Gengshen Wu, Min Yang
Abstract
Industrial retrieval systems often improve quality by enlarging embedding candidate pools or adopting complex training pipelines. We show that, under a sufficiently large but practical candidate pool, embedding recall exhibits diminishing returns, motivating a closer examination of reranker quality. On Chinese legal retrieval benchmarks, embedding top-300 reaches a high-recall operating region, while relevant evidence may still be ranked below the output cutoff. We compare hard-negative strategies for reranker training, including embedding-mined, self-mined, and dynamic-negative variants, under aligned retrieval protocols that evaluate reranking after candidate retrieval. Among the evaluated staged configurations, an embedding-to-self-mined pipeline provides a favorable trade-off; additional dynamic stages do not improve all held-out metrics. The selected pipeline scales from a 0.6B to a 4B reranker. Relative to the native 4B reranker, the trained 4B Stage2 model improves reported metrics on the STARD and CSAID benchmarks and on an internal development set, and is preferred in 72.4\% of non-tied comparisons on 410 de-identified, reconstructed internal operational-query inputs in a latency-sensitive offline preference evaluation. These results support evaluating the retrieval and reranking stages separately when selecting an optimization target for industrial retrieval systems.