MedVA system improves medical volume visualization with neuro-symbolic reasoning
MedVA: An End-to-End Neuro-Symbolic Agentic System for Medical Volume Visualization
GraphicsComputer Vision and Pattern RecognitionHuman-Computer Interaction
Summary
Medical images often contain complex 3D data that doctors need to explore by focusing on important parts. Traditional methods need experts to guess the best way to show these parts, which takes time and skill. The researchers created MedVA, a system that understands doctor instructions better by combining language models with medical rules, finds important regions using specialized models, and shows them clearly in 3D. This helps users, even those less experienced, get meaningful and reliable visualizations faster.
What this means in practice
- •For medical imaging teams: Create clearer 3D visualizations of key anatomical regions by automatically interpreting doctors’ natural language requests with clinical rules and segmentation models.
- •For radiology software developers: Integrate multi-agent neuro-symbolic approaches to improve medical volume viewers for faster and more reliable interpretation of imaging data.
Authors
Haill An, Suhyeon Kim, Minjun Kang, Eunwoo Lee, Bin Sheng, Lei Bi, Younhyun Jung
Abstract
Medical volume visualization requires selecting regions of interest (ROIs) and carefully controlling their relative visual emphasis according to a given clinical intent. Implementing these decisions in conventional workflows demands substantial clinical and visualization expertise and often involves trial-and-error optimization. Recent agentic systems have introduced natural-language interaction and autonomous visualization operations but largely rely on MLLM-based inference throughout the workflow. Although MLLMs encode broad medical knowledge and provide strong reasoning capabilities, such inference may be suboptimal for medical volume visualization, potentially leading to clinically incomplete interpretations of user requests and unreliable ROI identification and visualization optimization. In this work, we present MedVA, an end-to-end neuro-symbolic agentic system for medical volume visualization that addresses these limitations through three complementary agents. The neuro-symbolic intent formulation agent refines MLLM-based interpretations of natural-language requests through symbolic reasoning over established clinical knowledge, which provides more complete, clinically grounded ROI specifications than MLLM-only reasoning. The multi-model ROI identification agent directly identifies semantically specified ROIs in the original volume by leveraging complementary large-scale pretrained medical segmentation models. The objective-driven visualization optimization agent explicitly evaluates ROI visibility and occlusion in the original volume using a volume-based visibility objective. Extensive agent-level and system-level evaluations across diverse medical datasets and interaction scenarios support the effectiveness of the individual agents. A formative user study further indicates high usability and practical value among users with different levels of expertise.