Human in the loop AI cuts document annotation time by up to 80 percent
GAIDA: Generative AI for Human-in-the-Loop Document Annotation
Artificial Intelligence
Summary
Annotating documents by hand is slow and tiring. The authors designed a system called GAIDA that combines automated text reading with AI-generated suggestions that humans can easily fix or approve. This approach doesn’t try to remove humans but rather lets them work faster by reviewing and correcting these suggestions. Tests showed it reduced some tasks by over 80%, even though the AI wasn’t perfect. They also used the system to create a new Arabic receipt dataset by swapping out components easily.
What this means in practice
- •For document processing teams: Speed up manual labeling of scanned forms and receipts by integrating AI suggestions in an editable workflow.
- •For business data analysts: Create annotated datasets for specific languages and domains by adapting OCR and AI prompts within a flexible annotation system.
Authors
Islam Mesabah, Ali Elhosary, Priyabanta Sandulu, Darko Obradovic, Sebastian J. Vollmer
Abstract
Document annotation is a fundamental yet labor-intensive step in building document information extraction systems. While Optical Character Recognition (OCR) and Large Language Models (LLMs) can automate parts of this process, we argue that annotation systems should be judged not by how completely they remove human effort, but by how effectively they redirect it toward review and correction. We propose a modular human-in-the-loop architecture that integrates OCR-based extraction, LLM-based semantic pre-annotation, and human validation in a single editable workflow, and present \textbf{GAIDA}, a research prototype serving as its reference implementation. GAIDA supports interchangeable OCR engines, LLM backends, and prompt configurations, and enables stage-specific measurement of annotation effort. In an initial single-annotator, stage-specific study on the FUNSD scanned-form and WildReceipt retail-receipt benchmarks, OCR assistance reduced region-creation and transcription time by up to \textbf{80.9\%}, and reviewing LLM pre-annotations reduced semantic-labeling time by up to \textbf{73.7\%}, despite imperfect pre-annotations of 78.86\% and 86.98\% Micro F1 on FUNSD and WildReceipt, respectively. Finally, we reuse the same implementation, adapting only OCR and prompt configuration, to build and openly release \textbf{AraReceipt}, a manually annotated dataset of 100 Arabic receipt images labeled with 25 key-information classes. The results indicate that imperfect automated outputs provide practical value when presented as editable suggestions under human control.