Displacement Preserving Relational Distillation for Robust Medical Segmentation
2026-07-06 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors address the challenge of accurately segmenting 3D medical images despite differences in anatomy and high computing demands. They introduce Displacement-Preserving Relational Distillation (DPRD), a method that helps smaller models learn important orientation and scale details from larger models while focusing on key anatomical areas. By integrating DPRD into an existing segmentation tool, they achieve better results with much less computational power. Their approach works well on standard medical image datasets and keeps important structure details intact even with fewer resources.
3D medical segmentationknowledge distillationanatomical variabilitylatent anatomical trajectoriesvector alignmentnnU-NetDice scoreMedNeXtmodel compressionclinical imaging
Authors
Zhicheng Ding, Xinyu Chu, Jung Im Choi, Qing Tian, Tianyu Shi, Xiaoqian Jiang, Lijing Zhu, Qizhen Lan
Abstract
Accurate 3D medical segmentation is limited by anatomical variability and high computational costs. While knowledge distillation (KD) offers a route for model compression, conventional methods often fail to preserve complex structures and are overwhelmed by background noise. We propose Displacement-Preserving Relational Distillation (DPRD), which distills latent anatomical trajectories via vector based alignment to preserve the orientation and relative scale of the teacher's manifold, and prevents signal dilution by anchoring distillation in task-relevant structures. Integrated into nnU-Net, DPRD outperforms established baselines on ISLES 2022 and AMOS 2022 benchmarks. Notably, on the AMOS dataset, DPRD achieves a Dice score of 85.46%, edging out the high-capacity MedNeXt teacher while significantly reducing boundary errors. Despite utilizing only ~5% of the teacher's parameters and ~3% of its FLOPs, our approach maintains high structural consistency. This provides a robust, efficient solution for deploying high performance segmenters in resource-constrained clinical environments. Code: https://github.com/ClinicaAlpha/DPRD-3D-MedSeg