Question evidence supervision improves chunking for retrieval tasks
QESChunker: A Single Objective Unifies Overlapping and Non-Overlapping Chunking for RAG
Information RetrievalComputation and Language
Summary
Chunking is about breaking up documents into smaller parts so that retrieval systems can find relevant information more easily. Most existing ways to do this rely on simple rules, but the authors introduce QESChunker, a method that learns to create these chunks from question and evidence pairs. This new method treats chunking as a way to maximize how useful chunks are for answering questions, and it works for both overlapping and non-overlapping chunks. Their approach performs better than existing methods on various datasets and helps build smarter document indexes for retrieval tasks.
What this means in practice
- •For search engineers: Create document indexes that better capture relevant chunks for answering user queries by training on question and evidence examples.
- •For knowledge base managers: Build more effective chunked indexes that improve evidence retrieval quality for automated question answering systems without test-time query access.
Authors
Yifan Zhao, Qianyang Zhao, Xinyu Guan, kai wei, Yuming Deng
Abstract
Chunking is a fundamental preprocessing step in retrieval-augmented generation (RAG), yet most existing chunkers rely primarily on heuristics. We introduce the Question--Evidence-Supervised Chunker (QESChunker), to our knowledge the first method for constructing static chunk indices from question--evidence supervision. QESChunker constructs document-specific chunk indices by optimizing over human-annotated or LLM-generated question--evidence pairs, without access to held-out test queries. We formulate non-overlapping and overlapping index construction through a unified question--evidence utility-maximization objective, turning chunking into a supervised combinatorial optimization problem. Under the non-overlap constraint, the objective decomposes over chunks and can be optimized exactly in polynomial time by dynamic programming. When overlap is allowed, the objective becomes monotone submodular; exact optimization is NP-hard, but our greedy achieves a $(1-1/\sqrt{e})$-approximation. Across two self-constructed and three public datasets, both variants outperform existing chunking methods in complete-evidence recall in nearly all evaluated settings, under the same overlap regime and a fixed hybrid retriever. These results recast chunking as supervised, budget-aware index construction for evidence retrieval.