Walk Before You Run: The Importance of Data Exploration for Data Analysis Agents
2026-08-17 • Databases
DatabasesArtificial Intelligence
AI summaryⓘ
The authors explain that when using large language models (LLMs) to analyze messy spreadsheets, it's important not just to get the final answer right but also to understand the data well first. They call this early step 'Data Exploration,' which means figuring out how the spreadsheet is organized, what each column means, and spotting any data problems. Current tools often skip this step, so the authors made it a focus and created benchmarks to test how well systems understand the data structure. Their findings show that better Data Exploration helps improve final results and should be a key, reviewable part of data analysis with LLMs.
Large Language ModelsData ExplorationSpreadsheetsData AnalysisSchemaSemantic RolesData QualityBenchmarkHuman-in-the-loopDownstream Tasks
Authors
Yike Yuan, Virum Ranka, Tina Lasisi, Lin Ma
Abstract
LLM-based data-analysis tools are increasingly used to help users analyze messy spreadsheets and workbooks, from answering questions over uploaded files to generating code, summaries, and visualizations. These systems are often evaluated by the correctness of their final downstream answers. However, reliable data analysis also depends on an earlier step: understanding what the dataset contains before solving the requested task. For complex workbooks, this Data Exploration step includes identifying the logical tables behind physical sheets, interpreting column semantics, recovering keys and relationships, and detecting quality issues. In current tools and benchmarks, this step is usually left implicit, creating a gap between downstream task performance and the dataset understanding needed for reliable, human-checkable analysis. Our key contribution is to identify this overlooked gap, make Data Exploration a first-class evaluation target, and show through downstream experiments that stronger Data Exploration support improves task performance. To evaluate dataset understanding directly, we introduce two benchmark settings: a real multi-sheet workbook benchmark based on a Vitamin D study dataset, and an extension of DSBench with schema-fixed Data Exploration artifacts. In both settings, systems are evaluated by the quality of a structured artifact capturing tables, columns, semantic roles, relationships, and profiling signals. Our results show that strong LLMs and data-analysis agents still miss important logical structure even when they read spreadsheet content. Furthermore, explicit Data Exploration support often improves downstream correctness, suggesting it should be treated as a first-class, inspectable stage in LLM data-analysis workflows and a natural human-in-the-loop checkpoint where domain experts can review and correct the artifact before downstream analysis proceeds.