DeepTable improves understanding of complex tables in AI models
DeepTable: Structural Attention Biases and Tree Path Encoding for Hierarchical Table Understanding
Computation and Language
Summary
Large language models usually read tables by turning everything into a simple list of words, which loses the table’s complex structure like multi-level headers. The authors propose DeepTable, which helps these models better understand table layouts by adding special signals that show which parts of a table share rows or columns. They also represent each table entry by tracing the path of its headers, keeping track of its place in a multi-layer structure. Tests show that adding DeepTable improves performance on several tasks where models answer questions about tables, making AI better at understanding hierarchical table data.
large language modelstable understandinghierarchical headersattention mechanismstructural biasesparameter-efficient fine-tuningTree Path EncodingStructural Attention BiasTable question answeringmulti-level tables
Authors
Jyun-Ying Yen, Cheng-Kuan Lin, Yu-Chee Tseng
Abstract
Large language models (LLMs) have demonstrated strong performance in table understanding. However, they typically process table content and headers as linearized token sequences. This representation weakens the two-dimensional and hierarchical structural relationships encoded by multi-level row and column headers. Existing parameter-efficient fine-tuning methods incorporate basic row and column information but do not explicitly capture the rich structural dependencies induced by hierarchical table headers. We propose DeepTable, a structure-aware approach for table understanding with LLMs. DeepTable comprises two complementary components. Structural Attention Bias (SAB) introduces learnable biases into the attention logits to explicitly represent whether pairs of table tokens share the same row or column. Tree Path Encoding (TPE) represents each table token using the ancestor paths of its row and column headers, preserving its position within the multi-level table structure. We integrate DeepTable with TableLoRA (He et al., 2025) to inject structural information into parameter-efficient adaptation. Across three LLM backbones, DeepTable consistently improves the corresponding TableLoRA baselines on three table question answering benchmarks, achieving average gains of 7.42 points on HiTab, 3.23 points on WikiTQ, and 2.01 BLEU points on FeTaQA. These results demonstrate the effectiveness of the proposed structural biases across different LLM backbones.