TABVERSE: Benchmarking Cross-Format Table Understanding in LLMs and VLMs
2026-06-08 • Artificial Intelligence
Artificial IntelligenceComputation and LanguageInformation Retrieval
AI summaryⓘ
The authors created TABVERSE, a special test set where the same table content is shown in different formats like HTML, Markdown, LaTeX, and images. This helps them study how the format of a table affects how well language models can understand and answer questions about it. They found that models usually do better with text-based tables than with pictures of tables, but the best format can change depending on the task and model. HTML tables often work best, while tasks needing detailed structure understanding or LaTeX reconstruction are harder for the models. Overall, the study highlights that how a table is shown matters a lot for evaluating these models.
Large Language Models (LLMs)Vision-Language Models (VLMs)Table representationHTMLMarkdownLaTeXQuestion Answering (QA)Structural Understanding Capability (SUC)Structure Reconstruction (SR)Multimodal benchmarks
Authors
Momina Ahsan, Sarfraz Ahmad, Ming Shan Hee, Roy Ka-Wei Lee, Preslav Nakov
Abstract
Large Language Models (LLMs) and Vision-Language Models (VLMs) are increasingly evaluated on table reasoning tasks, but the role of table representation remains under-explored. In practice, the same table content may appear in different structural formats, such as HTML, Markdown, and LaTeX, or as rendered images. However, existing evaluations often let content, format, layout, and modality vary together, making it difficult to isolate representation effects. We introduce TABVERSE, a controlled multimodal table benchmark that aligns the same table content across multiple structural formats and rendered images, with question category and difficulty tags. This design enables systematic evaluation of representation effects while holding table content fixed. We evaluate LLMs and VLMs across three tasks: Question Answering (QA), Structural Understanding Capability (SUC), and Structure Reconstruction (SR). Our results show that representation choice substantially affects table understanding. Models generally perform better with structured text than with rendered images, but the size of this gap depends on the task, model, and format. HTML is often the most robust text format, while row-sensitive structural tasks and syntactically usable LaTeX reconstruction remain challenging. These findings show that table representation is a key factor in reliable table evaluation.