Chart agents improve long-tail chart reasoning with self-evolving tools
ChartEvo: Self-Evolving Agent Harnesses for Long-Tail Chart Reasoning
Artificial IntelligenceComputer Vision and Pattern Recognition
Summary
Charts come in many different types, and computer models understand some common ones well but struggle with rare or unusual charts. The authors introduce ChartEvo, a system that helps chart-reading agents improve themselves by learning from their mistakes with difficult charts. It builds specialized skills and tools over time to better handle tricky or uncommon chart types. ChartEvo showed big accuracy improvements on datasets containing these hard-to-understand charts.
What this means in practice
- •For business analysts: Improve automated interpretation of rare or complex charts in business reporting for more accurate data insights.
- •For data visualization software developers: Enhance data visualization tools with self-improving agents that adapt to diverse chart types without manual reconfiguration.
Authors
Qi Wang, Peijie Wang, Jun Cao, Haoyang Tong, Xiao-Hui Li, Fei Yin, Cheng-Lin Liu
Abstract
Multimodal large language models (MLLMs) have substantially advanced chart understanding and reasoning, yet their performance remains highly uneven across chart types. While modern MLLMs perform well on common charts, their accuracy drops sharply on long-tail charts with diverse visual structures and reasoning requirements. Tool-augmented agents can improve complex chart reasoning by introducing specialized visual capabilities, but manually engineering strategies and tools for the heterogeneous and evolving long tail is difficult to scale. We introduce \textbf{ChartEvo}, a recursive self-improvement framework that enables chart agents to adapt to underperforming chart types through experience-driven harness-level self-improvement. ChartEvo jointly evolves reusable chart-specific Skills and executable Tools, while learning reusable local tool-call chains to compose evolved capabilities and control the inference overhead of tool-augmented reasoning. On long-tail charts from ChartBench and ChartX, ChartEvo improves accuracy over its base MLLM, Qwen3.7-Plus, by \textbf{45.6\%} and \textbf{32.5\%}, respectively, demonstrating the effectiveness of recursive self-improvement for long-tail chart adaptation.