Graphionale: How Graph Visualizations of LLM Rationales Affect Human Decision Making
Human-Computer Interaction
Summary
The authors studied how people understand explanations (rationales) from Large Language Models by comparing traditional text-based explanations to new graphical ones that show arguments as node-link maps. They found that graphical rationales help people trust verbal reasoning more but also feel harder to process, while for visual reasoning tasks, graphical explanations are more enjoyable but actually reduce trust accuracy. The authors highlight that the best explanation format depends on the kind of reasoning task, and what feels best to users isn’t always what leads to better decisions. Their work provides guidance on designing AI explanations to better support human decision-making.
Authors
Xinru Wang, Zhexuan Ma, Ming Yin, Shuai Ma, Thomas W Malone
Abstract
Large Language Models (LLMs) are increasingly equipped with augmented reasoning capabilities to generate rationales that support human decision-making. Yet these text-dense rationales often impose substantial cognitive burdens. Building on a formative co-design study that identified user preferences for non-linear reasoning representations, we developed Graphionale as a testbed for empirically studying argument-map-style rationale visualization. This system transforms linear LLM rationales into interactive, multi-level graphs. It explicitly structures logical relationships (e.g., conclusions, premises, support, and objections), while further extracting entities and relations within each statement to construct condensed node-link representations. We conduct a large-scale online user study (N = 204) to examine when graphical rationales are more effective than textual ones, across varying task modality (verbal vs. visual reasoning), rationale format (textual vs. graphical), and question difficulty (easy vs. hard). Our results show that graphical rationales do not help uniformly: they improve trust calibration for verbal reasoning yet feel more cognitively demanding and less satisfying; for visual reasoning, they impair calibration yet feel more engaging and helpful. In each modality, the format that better supports calibrated decisions is not the one users prefer, highlighting that matching rationale format to task modality is key to effective AI explanation design. Our findings contribute empirical design knowledge about when and how graphical rationales support human decision making, and inform the next-generation reasoning-aware AI interfaces.