Information extraction methods vary by document type for accuracy and energy
The Right Information Extraction Pipeline Depends on the Document: Accuracy-Energy Trade-offs for Small, Local Models
Artificial IntelligenceComputation and Language
Summary
Deciding how to extract information from documents depends on what kind of document it is. The authors show that using image-based or text-based models works better for different layouts and that batching document processing greatly saves energy without losing accuracy. They also find that using cheaper text parsing on simple documents is more efficient than complex image analysis, but for intricate layouts, vision-language models perform best. These findings help guide using local, privacy-friendly methods that balance power use and accuracy.
What this means in practice
- •For enterprise software teams: Choose efficient local document processing methods tailored to contract or form layouts to save energy and preserve privacy on-premise.
- •For document management providers: Develop document handling features that switch between lightweight text parsing and vision-language models depending on document complexity to optimize costs.$Commercial implications: Enables selling smarter, energy-efficient document extraction solutions customized by document type.
Authors
Christoph Walser, Mauricio Fadel Argerich, Jonathan Fürst
Abstract
Whether an information extraction pipeline should process page images or parsed text depends on the document, and the answer flips across the layout spectrum. We study this trade-off under a constraint that rules out (closed) cloud services: privacy-sensitive documents processed on-premise by small ($\le 8\mathrm{B}$ parameter) text-only and vision--language models, evaluated on both accuracy and energy over a design space spanning input representation, model family, and inference configuration. Benchmarking on the near-plain-text Kleister-NDA contracts and the layout-rich VRDU forms, we find that batching is the dominant energy lever, cutting energy per page by 38-85% at no cost in accuracy, while FP8 quantization saves 27-32% when requests are served one at a time but less than 1mWh per page (9-19%) once batching is applied. Preprocessing dominates what remains: neural OCR costs $17\times$ more energy per page than classical OCR and never reaches the Pareto frontier. Which representation wins flips with the type of document: vision--language models on layout-rich documents and small text-only models with a cheap parser on near-plain text, where they are both more accurate and cheaper than any vision--language configuration. Our work yields concrete guidelines for energy-efficient, privacy-compliant local information extraction.