Augmented reality shows AI decisions in the real world
Seeing through the Eyes of AI: Situated Explainability in Augmented Reality
Human-Computer Interaction
Summary
AI systems often explain their decisions using flat screens that can be hard to connect with real-world objects. The authors suggest using augmented reality to display AI explanations directly in the physical space where decisions happen, helping users understand AI behavior more naturally. They demonstrate how current AI explanation methods can work with augmented reality and share insights from people using the approach.
What this means in practice
- •For industrial maintenance teams: Visualize AI diagnostic explanations directly on machinery to speed troubleshooting and reduce errors in complex physical environments.
- •For smart building facility managers: Provide on-site AI insights for environmental monitoring systems through AR to improve decision-making in managing building operations.
Authors
Ana Stanescu, Lucchas Ribeiro Skreinig, Tobias Langlotz, Stefanie Zollmann, Peter Mohr, Dieter Schmalstieg, Mark Billinghurst, Denis Kalkofen
Abstract
Explainable Artificial Intelligence (AI) enables humans to understand and interpret decisions of AI models. Instead of having a black box, explainability supports humans in understanding AI models' behavior. Existing explainable AI approaches often present explanations on 2D displays using pre-recorded data, requiring users to relate the displayed information back to the physical objects and real world locations involved in a model's decision. Users are forced to decouple data exploration and capture from AI model interpretation. For AI systems that work within physical environments, this separation can make explanations difficult to interpret in context. We propose using Augmented Reality (AR) to enhance the understanding of AI models by enabling spatial explainability information directly in a user's workspace, in real time, as they explore the world. We show how known explainability methods can be applied in AR and provide insights into user experiences with such an application.