Self-correcting vision models improve table question answering accuracy
Cycle-of-Thought: Self-Evaluative Iterative Reasoning for Table Visual Question Answering in Document Images
Computer Vision and Pattern RecognitionArtificial IntelligenceComputation and Language
Summary
Answering questions about tables in images is tricky for current vision-language models because they often make mistakes in understanding the layout or logic. The authors introduce a new method called Self-RefineVQA that helps these models correct their own answers by checking and revising them without needing extra training or labeled data. This approach works like a thinking loop where one model generates an answer, critiques it, and then tries to improve it. Tests show that even smaller models can reach the performance of much bigger ones using this method, making it easier and cheaper to get accurate answers from table images.
What this means in practice
- •For enterprise analytics teams: Enhance automated extraction and reasoning from tables in business documents without costly labeled data or retraining large models.
- •For multilingual software developers: Build lightweight applications that accurately answer questions about tabular data in various languages using iterative self-correction.
Authors
Somraj Gautam, Gaurav Harit
Abstract
Recent advances in Vision-Language Models (VLMs) have significantly improved Visual Question Answering (VQA) performance, yet their reasoning over structured content such as tables remains inconsistent, often plagued by spatial misalignment and logical hallucinations. Existing approaches for improving table VQA typically rely on architectural modifications or expensive supervised fine-tuning (SFT), which impose high barriers to entry due to the need for large-scale, expert-annotated datasets and significant computational resources. In this work, we propose Self-RefineVQA, a training-free, inference-time self-correction framework that improves table VQA performance without additional labeled data or model updates. Inspired by human revision behavior, our method enables a single VLM to iteratively act as a generator, critic, and refiner, forming a closed self-feedback loop that corrects common reasoning errors through prompt-guided re-evaluation. Experiments on tabular VQA benchmarks demonstrate that Self-RefineVQA yields consistent accuracy gains in monolingual and cross-lingual understanding, with a particularly significant boost for small models, enabling lightweight models (e.g., 2B-7B parameters) to rival the performance of significantly larger counterparts. Furthermore, we provide a comparative complexity analysis demonstrating that while inference cost increases linearly, our framework offers superior data efficiency and a lower cumulative cost by eliminating the prohibitive annotation bottleneck of human labeling required for fine-tuning. Our results highlight the potential of prompt-based self-improvement as a scalable, accessible paradigm for reliable table reasoning in VLMs.