When Is Complex Chunking Worth It? A Multi-Objective Evaluation of Chunking Methods at Scale
2026-08-17 • Information Retrieval
Information Retrieval
AI summaryⓘ
The authors studied different ways to break up long documents into smaller pieces (chunking) for search systems that use dense embeddings. They found that more complex chunking methods don't always improve search results and can be more costly to run. Instead, the best chunking method depends on the specific search model, dataset, and goals. They also showed that similar chunking methods might perform alike but have very different resource costs. Overall, they suggest choosing chunking methods by balancing both accuracy and system costs.
dense retrievaldocument chunkingembedding modelsretrieval effectivenessindexing throughputquery latencymemory usageembeddinginformation retrievalsystem-level costs
Authors
Laura Caspari, Kanishka Ghosh Dastidar, Michael Dinzinger, Jelena Mitrović, Michael Granitzer
Abstract
Dense retrieval is commonly evaluated on benchmarks that represent each document with a single embedding, even though real-world retrieval systems often index long documents that require chunking. In these settings, the chosen chunking method not only affects retrieval quality, but also indexing throughput, query latency, and memory usage. Prior comparisons of chunking strategies have mainly focused on retrieval performance, leaving operational trade-offs underexplored. To address these issues, we evaluate eight representative chunking strategies across two scalable corpora, three embedding models, and multiple corpus sizes, measuring both retrieval effectiveness and system-level costs. Our results show that computationally expensive methods rarely provide consistent gains over simpler chunking. Instead, the best performing strategy depends on the embedding model, dataset, corpus size, and target retrieval metric. Methods with similar performance can also differ substantially in operational cost, showing that chunking should be seen as a multi-objective design decision.