System handles degraded historical archives with noisy ocr errors
Reading Beyond OCR Noise: Robust Cross-Lingual RAG for Legislative Archives
Computer Vision and Pattern RecognitionArtificial IntelligenceInformation Retrieval
Summary
Old government papers from India are hard to read because the pages are damaged and scanned with different languages and tricky layouts. The authors created a system that combines reading these mixed scripts with a smart, back-and-forth process to fix mistakes caused by bad text recognition. This approach helps the system better find information from the messy scanned documents. They tested it on a large collection of 200,000 pages and it worked better than usual methods that just use plain text.
What this means in practice
- •For government archives teams: Automate extracting accurate information from degraded, multi-language scanned parliamentary documents to improve access to historical records.
- •For legal document processors: Enhance handling of complex, noisy legislative documents with mixed scripts by reducing OCR-related errors during information retrieval.
Tested on one dataset.
Authors
Radha Krishna Deshpande, Ajoy Mondal, Akash Manna, C V Jawahar
Abstract
India's historical public archives present severe computer vision and document image analysis challenges: decades of parliamentary records survive only as degraded, multi-column scanned pages featuring mixed scripts, complex layouts, and dense visual noise where OCR failure is the primary obstacle to information retrieval. We present an end-to-end framework that couples vision-based layout segmentation and multi-script OCR with an agentic retrieval-augmented generation (RAG) pipeline. Our approach specifically addresses how visual degradation and layout fragmentation propagate into text extraction errors, using an iterative agentic loop to navigate around OCR noise and layout ambiguities before generating answers. Evaluated on the 200,000-page Punjab Vidhan Sabha archive, our system demonstrates robust handling of historical document degradation, outperforming standard text-only baselines.