Conversational tool helps users correct misunderstandings of AI explanations
Explanation Navigator: Rectifying Out-of-Scope Human Interpretations of Leaky AI Explanations through Conversational Guidance
Human-Computer Interaction
Summary
People often misunderstand or fill in gaps when AI explanations don’t provide all the information needed. The authors show that such explanations can leak complexity, causing users to guess incorrectly without realizing it. They created Explanation Navigator, a chat-based system that detects when users want more information and guides them to a better understanding. Tests with many users showed it helped people notice and fix their mistaken ideas about AI behavior.
What this means in practice
- •For ai product designers: Integrate conversational guidance to improve user understanding of AI system explanations and reduce misinformation from oversimplified explanations.
- •For customer support teams: Use conversational explanation tools to clarify AI decisions for users, helping customers resolve issues caused by misunderstood AI outputs.
Authors
Yueqing Xuan, Kacper Sokol, Danula Hettiachchi
Abstract
As explanations of artificial intelligence systems proliferate, their recipients must grasp not only what they convey but also recognise what they cannot. We conducted an interview study with nine participants to examine how explainees reason when their information needs exceed the scope of available explanations. Participants often unwittingly confabulated explanatory insights when relevant information was missing from the explanations, not recognising the inherent limitations thereof. We characterise such explanations as leaky explanations -- simplifications that strive to hide complexity yet whose correct interpretation hinges on understanding of the concealed details. To address out-of-scope interpretations we propose Explanation Navigator: a conversational interaction framework that detects mismatches between users' information needs and explanations' content, elucidating pertinent yet implicit details and providing complementary explanations for unmet information needs. An online study with 316 participants showed that our approach allowed explainees to recognise and rectify confabulated explanatory insights, guiding them towards developing correct understanding.