Framework measures explanation quality over time in heart ultrasound AI
A Quantitative Evaluation Framework for Temporal Explainability in Echocardiographic Video Segmentation
Computer Vision and Pattern RecognitionMachine Learning
Summary
Deep learning models can identify heart structures in ultrasound videos, but it’s unclear how well explanations show what the model focuses on over time. The authors created a way to measure how consistent and meaningful these explanations are throughout the video sequence. They found that some model parts produce more stable explanations than others, but common methods can’t always tell if changes in explanations are meaningful or just noise. This work helps guide better explanation tools for medical video AI.
What this means in practice
- •For medical imaging developers: Improve reliability of explanation tools for heart ultrasound segmentation models by measuring how explanations evolve over time.
- •For video ai engineers: Create temporally consistent explanation methods for models processing sequential visual data to better understand feature changes over time.
Tested on one dataset.
Authors
Jiyoo Noh, Jonathan H. Chan
Abstract
Deep learning has achieved state-of-the-art performance in echocardiographic video segmentation, with an increasing number of models incorporating temporal information. However, quantitative evaluation of temporal explainability remains largely unexplored. We propose a quantitative framework for evaluating Grad-CAM explanations using four complementary metrics measuring temporal consistency, saliency motion, anatomical overlap, and temporal overlap. Using EchoNet-Dynamic, we compare a baseline 2D U-Net with ConvLSTM U-Net models trained across multiple temporal strides. While segmentation performance remained comparable across all models, intermediate ConvLSTM explanations exhibited substantially lower saliency consistency and greater centroid motion than final prediction explanations. Temporal Bottleneck explanations were significantly more stable than Encoder Bottleneck explanations across all strides, while final ConvLSTM Decoder3 explanations were broadly comparable to those of the 2D U-Net. Importantly, conventional frame-wise explanation metrics cannot determine whether variation in intermediate explanations reflects meaningful temporal feature evolution or explanation instability. These findings establish a preliminary quantitative framework for temporal explainability and motivate temporal-aware XAI methods that explicitly account for evolving representations in medical video models.