Large language models struggle to answer questions from flawed tables accurately
Understanding Errors in LLM-Based Question Answering over Imperfect Tables
Computation and Language
Summary
When large language models answer questions using tables that have mistakes, their answers can be wrong. The authors studied how changing the order of table rows and showing where errors were affects the model's ability to find and fix these mistakes. They found that simply marking errors isn’t enough to get correct answers; actually fixing the mistakes in the table greatly improves accuracy. They propose a method called GBDI that helps models find and handle errors better, leading to more accurate answers. This work shows that both finding and fixing errors in tables are important for reliable question answering.
What this means in practice
- •For data engineers: Improve automated systems that extract answers from imperfect datasets by incorporating error detection and repair steps identified in this study.
- •For business analysts: Use improved question answering tools that handle errors in tabular reports to get more accurate insights without manual data cleaning.
Authors
Baowen Zhang, Wei Fan, Ruman Wang, Hangting Ye
Abstract
We investigate error discovery and handling in question answering over imperfect tables through controlled studies across three large language models (LLMs) on human-reviewed RADAR-T examples. Answering questions over these tables requires handling errors that can affect the answer. We vary row order and compare original, error-marked, and repaired tables to test whether discovery depends on where errors appear and whether providing their locations is sufficient for accurate question answering. First, reordering rows changes error discovery even when the table contents and gold answer remain unchanged. Complete discovery is higher for back than front placements and, averaged over the tested mean positions, for compact than widely spaced layouts. Second, providing verified error locations alone is insufficient for accurate QA, leaving a substantial accuracy gap between error-marked and repaired tables. Providing tables with human-reviewed repairs already applied raises code-assisted QA accuracy by 39.0-59.1 percentage points over the error-marked tables across the three systems. GBDI, a simple workflow, puts these findings into practice by combining error discovery across shuffled table views with explicit guidance for verifying and handling the reported errors. On RADAR-T, GBDI raises observed QA accuracy by 3.8-18.5 percentage points over a code-agent baseline across five systems. These results highlight the importance of both reliable error discovery and effective error handling in question answering over imperfect tables. Our anonymous repository is available at https://anonymous.4open.science/r/GBDI-ICLR-2027-85BD/