Interactive system improves chart making from imperfect text requests
VisInteract: Towards Dynamic Interactive Text-to-Visualization under Imperfect Queries
Artificial Intelligence
Summary
People often ask computers to create charts from text instructions, but these requests can be unclear or incorrect. The authors propose VisInteract, which helps the system talk back and forth with users to better understand what chart they really want. They also created a test setup to see how well this interactive process works when the instructions are imperfect. Their improved method, Vis-MCTS, uses smart search techniques to ask clarifying questions and learn from feedback, leading to much better final charts. This approach outperforms previous non-interactive and interactive methods by a wide margin.
What this means in practice
- •For data visualization developers: Create interactive tools that refine user chart requests through conversation, handling unclear or incorrect text inputs better.
- •For business intelligence teams: Improve dashboard generation by dynamically clarifying ambiguous queries to ensure accurate data representation.
Authors
Wenxin Xu, Jinwei Lu, Hwanhee Kim, Chen Jason Zhang, Xiao-Yong Wei, Haoyang Li, Yuanfeng Song
Abstract
Real-world visualization requests are routinely ambiguous, incomplete, or factually incorrect, yet existing Text-to-Visualization (Text-to-Vis) systems assume well-specified inputs and produce charts in a single pass. When queries are imperfect, a system must \emph{interact} with the user to recover the true intent, but no benchmark or method supports this dynamic process. We introduce \textbf{VisInteract}, a new paradigm that reframes Text-to-Vis as interaction-driven intent recovery, and \textbf{VisInteract-Bench}, to our knowledge, that is the first benchmark for dynamic interactive Text-to-Vis, featuring controlled imperfection injection, a leakage-controlled User Agent for realistic multi-turn feedback, and dual-perspective (code and chart) automated evaluation. On the algorithmic side, we propose \textbf{Vis-MCTS}, a Monte Carlo Tree Search (MCTS) enhanced method, introducing improvements over classical MCTS, that \emph{Progressive Widening} to tame the unbounded tool-argument space in tree search, \emph{cross-rollout information sharing} so clarifications and critiques benefit the entire search tree, and \emph{Dimension-Aware Reward Decomposition} that routes scalar user feedback along data-fidelity, visual-design, and intent-alignment dimensions to resolve credit assignment across heterogeneous actions. Extensive Experiments across two LLM backbones show that Vis-MCTS consistently outperforms all Text-to-Vis baselines, improving end-to-end task success by $13.40\%$--$16.27\%$ over the strongest interactive baseline and by more than $5\times$ over non-interactive ones.