PrismAlign improves table text extraction with multi-view alignment
PrismAlign: Prior-Steered Multi-View VLM Alignment for Hallucination-Robust Table OCR
Computer Vision and Pattern Recognition
Summary
Extracting information from tables in images often leads to mistakes in the structure and made-up content. The authors created PrismAlign, a system that combines different views of the same table and uses rules about how tables should look to improve accuracy. It separates fixing the layout from reading the cell content, and uses a statistical method to find the best alignment. Tests show PrismAlign reduces errors and works better than past methods on several datasets.
What this means in practice
- •For document digitization teams: Improve reliability of automated table extraction from scanned documents by reducing layout and content errors in OCR outputs.
- •For data integration specialists: Enhance data accuracy in workflows that combine information from tables in diverse document images by resolving ambiguities through multiple visual perspectives.
Authors
Guangyi Liu, Qianjun Huang, Boyu Hou
Abstract
Table extraction suffers from frequent structural errors and semantic hallucinations. We propose PrismAlign, a multi-VLM framework aligning diverse visual perspectives to resolve ambiguity. It integrates priors of table logic to assess output plausibility, decoupling structural alignment from cell content alignment. A Bayesian decision strategy maximizes alignment accuracy by exploiting the correlation between extraction errors and computable rule violations. Evaluated on open-source and custom VLMs, PrismAlign reduces hallucinations and achieves state-of-the-art performance on OmniDocBench 1.5, as well as on the table category of CC-OCR and PureDocBench.