The Role of Mixed and Augmented Reality in Medical Visualization: Literature Review and A Context-Aware Taxonomy
Graphics
Summary
The authors explain that while medical imaging helps doctors see inside the body without surgery, it's hard to fully understand these images because they are usually shown on flat screens. They explore how Augmented and Mixed Reality (AR/MR) can let users view and interact with 3D anatomy in more natural ways. The authors created a clear system to organize different AR/MR visualization methods based on factors like image type and how the virtual images relate to the real world. Their work helps guide better design choices for using AR/MR in medical settings.
Authors
Xinrui Zou, Mingxu Liu, Hongchao Shu, Ruixing Liang, Mathias Unberath, Alejandro Martin-Gomez
Abstract
The discovery and evolution of medical imaging technologies have enabled non-invasive visualization of internal anatomy that has be- come essential for supporting diagnosis, monitoring, and treatment. However, because medical imaging relies on complex physical processes and contrast mechanisms for image formation, imaging alone is not sufficient to enable humans to leverage the resulting in- formation fully. In addition, traditional methods to visualize the resulting information use two-dimensional displays to present three-dimensional anatomical structures. The introduction of Augmented and Mixed Reality (AR/MR) technologies offers an opportunity to provide valuable paradigms for medical imaging visualization, allowing users to observe, explore, and interact with anatomical infor- mation in more spatially intuitive ways. However, naive implementation without careful design considerations can lead to perceptual inconsistencies, potentially compromising utility and effectiveness. In this paper, we present a structured taxonomy of medical AR/MR visualization strategies aimed at providing clearer insight into how visualization design varies across clinical use cases. The taxonomy organizes techniques based on four core design components: image modality, data dimensionality, display technology, and clinical application. In addition, we introduce two critical dimensions that are often overlooked in the literature: visualization anchoring (the spatial relationship between virtual content and the physical world), and perceptual awareness (the use of visual cues to support spatial interpretation). Together, these components form a comprehensive taxonomy, offering a detailed framework for selecting appropriate visualization techniques in medical applications.