Chat assistant helps explore and modify software in 3d visualization
A Chat Assistant for Software Exploration in a 3D Software Visualization
Software Engineering
Summary
Understanding complicated software can be hard, so the authors built a chat assistant that works inside a 3D software viewer called ExplorViz. This assistant answers questions and changes the visualization based on what users say in natural language. People tried it out and found it useful for getting summaries and highlighting parts of the software, but changing software structure was harder and worked less well. The authors show that this assistant can make exploring software easier, though it needs better controls and user feedback for complex changes.
What this means in practice
- •For software developers: Use natural language to explore and modify software structures within a 3D visualization tool to improve understanding and interaction speed.
- •For game designers: Interact with complex 3D game codebases by asking questions and triggering visual changes through conversational commands to streamline debugging and feature exploration.
Authors
Malte Hansen, Karim Issa, Wilhelm Hasselbring
Abstract
We present a chat assistant for interactive software exploration, embedded in the 3D software visualization tool ExplorViz. The assistant builds upon current Large Language Models (LLMs) and enables users to ask questions about the currently visualized software system and trigger actions that change the visualization through natural language. We integrate the chat assistant in our ExplorViz frontend using the CopilotKit libraries such that probabilistic LLMs are combined with deterministic and tool-based actions similar to implementations using the Model Context Protocol (MCP). The chat assistant is also enabled to restructure the software system by adding, removing, or modifying parts of the software system in the visualization. An empirical experiment with eleven participants evaluated both perceived comprehension support and the tool calls that were triggered by the chat assistant. Participants rated the assistant's generated summaries and explanations as largely correct. They also reported high usability for actions like highlighting entities in the visualization and the creation of new color themes. In contrast, open-ended chat-assisted software restructuring in the visualization showed mixed results. This suggests a need for stronger guardrails for the employed LLM and better incorporation of user feedback. Overall, the assistant was perceived as usable and promising for reducing interaction overhead during exploratory program comprehension tasks. We provide a video presenting the chat assistant's use and a reproduction package of the software system that was used in our evaluation.