Text to visualisation benchmark compares leading ai chart systems
VisCodeBench: a new benchmark dataset for Text-to-Vis reflecting real-world practice
Computation and LanguageHuman-Computer Interaction
Summary
Creating charts from written descriptions can help people understand data faster. The authors introduce VisCodeBench, a dataset that tests how well AI systems can turn text into visuals across different real-world scenarios. They found that while AI can often create basic charts correctly, none can perfectly match the example data, especially for harder cases. The study also shows that the way AI tools are designed around language models matters a lot, as does how fast they produce the charts.
What this means in practice
- •For business intelligence teams: Improve automated chart generation tools by using VisCodeBench to benchmark and select AI systems for real-world visualisation tasks.
- •For data journalism teams: Evaluate and choose AI-powered chart creation systems that best match complex data descriptions from diverse document types.
Authors
Hy Nguyen, Thomas James Elton, Ella Wilson, Andrew Reeson, Cecile Paris, Josiah Poon, Jonathan K. Kummerfeld
Abstract
Generating visualisations from natural language requests (text-to-vis) enables users to explore data more efficiently. While LLMs can complete simple visualisation requests, their performance on challenging ones is not well covered by existing benchmarks. We introduce VisCodeBench, an end-to-end text-to-vis benchmark with 2,672 samples from three different domains: educational, scientific, and general audience scenarios with varying levels of complexity. We benchmark three visualisation systems---Codex, Claude Code, and LIDA---paired with a range of open and commercial LLMs across seven metrics covering execution, chart content, and visual quality. While basic execution is near-solved for strong systems, no system comes close to reproducing the reference data, with the best displayed data similarity reaching only $0.81$ and falling to $0.69$ on our hardest subset for this metric. We further find that frontier models separate on chart correctness and efficiency, with ClaudeCode-Opus-4.7 improving chart correctness by around 5 points over Codex-GPT-5 while running roughly 20 seconds faster. Separately, we find that system design matters as much as the underlying model: despite both running on GPT-5, Codex clearly outperforms LIDA, showing that the surrounding prompting, execution, and refinement pipeline can shape outcomes as much as the model itself. Finally, latency remains a practical obstacle, as even the strongest systems take 39-78 seconds to produce results. VisCodeBench is available at \href{https://github.com/SAIPLL/VisCodeBench}{https://github.com/SAIPLL/VisCodeBench} and accessed via a leaderboard; the full dataset is kept private to prevent LLM training use.