Ragit enables better testing of document preprocessing for retrieval systems
RAGit: Turning Document Preprocessing into an Experimental Variable for RAG
Information Retrieval
Summary
Document preprocessing changes how search systems that help generate answers find the right information, but these changes are often chosen based on guesswork. The authors made RAGit, a tool that lets developers test different ways to prepare and split documents for these search systems under the same conditions. RAGit helps compare those methods fairly by controlling other settings and keeping track of where retrieved information originally came from. This makes it easier to understand how changing document preparation affects search results and answer quality.
What this means in practice
- •For search system builders: Evaluate how different document splitting and indexing methods affect retrieval quality in RAG pipelines to improve answer accuracy.
- •For enterprise knowledge teams: Compare document preprocessing approaches under consistent conditions to optimize internal retrieval systems for enterprise data.
Authors
Omar El Bachyr, Saad Ezzini, Tegawendé F. Bissyandé, Jacques Klein
Abstract
Document preprocessing choices can substantially affect retrieval in Retrieval-Augmented Generation (RAG) pipelines, yet they are often selected heuristically and are difficult to compare under consistent conditions. We present RAGit, an open-source experimentation workbench for the systematic and reproducible evaluation of document preprocessing choices through downstream retrieval. RAGit represents parsing, chunking, indexing, retrieval, and evaluation as configurable experimental stages, enabling controlled comparisons while the remaining downstream settings are kept fixed. It preserves provenance from retrieved chunks to source pages and maps heterogeneous retrieval units to a common page-level ranking for evaluation. The current implementation supports multiple parsers and chunking strategies, dense and late-interaction retrieval, reusable intermediate artifacts, and multi-configuration benchmarking. We demonstrate RAGit through controlled experiments comparing alternative preprocessing configurations under a fixed retrieval setup, illustrating how preprocessing can be treated as an explicit experimental variable. RAGit and its documentation are publicly available on GitHub (https://github.com/OmarElbachyr/ragit). A demonstration video is available at https://youtu.be/tsst-Bhfz4I.