Vibe Visualizing: How Visualization Novices Try (and Fail) to Generate and Interpret Visualizations with Conversational AI
2026-06-08 • Human-Computer Interaction
Human-Computer Interaction
AI summaryⓘ
The authors studied how people new to data visualization use ChatGPT to create and understand charts by talking to it. They found that while ChatGPT helps lower barriers, it can also produce mistakes and misleading visuals that users may not easily spot. By analyzing conversations with 20 beginners, the authors identified common errors, user behaviors, and trust factors. They also tested other AI models and saw different problem patterns. From this, they suggested ways to improve future AI tools for making and interpreting visualizations.
Conversational AIData VisualizationChatGPTUser StudyAI-generated VisualizationsThematic AnalysisHuman-AI CollaborationPrompting PatternsModel Failure ModesAI-assisted Visualization Systems
Authors
Sam Yu-Te Lee, Yun-Hsin Kuo, Chifang Chou, Matthew Ward, Xiwei Xuan, Kwan-Liu Ma
Abstract
Conversational AI has enabled users to generate and interpret visualizations through natural language, significantly lowering the technical barrier to entry. The increased accessibility brings visualization novices into data visualization, but also exposes them to misinformation and misinterpretations. We are motivated to examine what issues can arise in interactions with current conversational AI, whether visualization novices can recognize such issues, and how they respond to them. To examine these questions, we conducted a user study on ChatGPT with 20 visualization novices, collecting their conversation logs, semi-structured interview transcripts, and Likert-scale questionnaire responses. Through thematic analysis, we developed a codebook that covers AI execution compliance, issues of AI-generated visualizations, patterns of AI responses, and prompting patterns of users. We summarized four themes, including the quality of outcomes, recurring errors from ChatGPT, misuse by users, factors that affect user trust, confidence, and verification behavior, and human-AI collaboration dynamics. To demonstrate the generalizability of our codebook and findings, we replayed the initial user prompts on Gemini and Claude and compared the outcomes, which revealed distinct failure modes for each model. Based on the results of all analyses, we derive a set of design recommendations for future AI-assisted visualization systems. We conclude with discussions on literacy gaps, diverse human-AI collaboration dynamics, and implications for agentic visualization.