Charts improve data agents ability to analyze complex data visually

Charts as Epistemic Actions for Experience-Driven Agentic Data Analysis

Artificial Intelligence

Summary

People who analyze data often switch between numbers, tables, and charts to find important trends and anomalies. The authors found that data agents using text-based reasoning miss out on insights that charts provide. They created a set of tasks and showed that agents can do well using charts but don’t always choose to use them. Then, they developed a method to teach agents when and how to use charts effectively, improving their accuracy without changing the underlying AI models.

What this means in practice

  • For business intelligence teams: Improve automated data analysis tools by enabling agents to incorporate charts for better detection of trends and anomalies in structured data.
  • For financial analysts: Enhance AI assistants to provide visual evidence alongside textual summaries, supporting more insightful temporal and anomaly analysis.

Authors

Xinzhe Wang, Xudong Weng, Zang Li, Shaolei Zhang

Abstract

Data agents built on ReAct-style loops now execute complex analytical workflows over structured data, yet their intermediate reasoning still runs almost entirely through text. Human analysts instead move between numbers, tables, and charts, reading visual evidence to expose trends and anomalies that text summaries drop. We ask whether charts can serve as internal epistemic actions inside a data agent's reasoning loop, and how to teach an agent to use them. We first build \probe{}, a suite of 200 controlled tasks under four matched interface conditions, and find a persistent gap between what agents can read and what they choose to use. Agents score as well from self-generated charts as from numeric output, yet plotting access alone does not translate into consistent use. Task slices show that charts lead on temporal, pattern, and anomaly reasoning, while text leads on precise reading and conditional verification. We then propose \method{}, which distils two families of experience from conditioned trajectory rollouts. One family records how to select and verify analytical operations, and the other records when to invoke text, charts, or both. A trajectory judge scores outcomes and labels intermediate steps, and a validation gate admits only experience that improves held-out performance. On three held-out benchmarks with four LLM backbones, \method{} raises analytical accuracy and visual evidence quality over matched baselines, and backbones that never drew charts begin using them, all without touching model parameters.