A Comprehensive Survey of Medical Image Segmentation: Challenges, Benchmarks, and Beyond
2026-06-15 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial Intelligence
AI summaryⓘ
The authors review important methods used to automatically identify and outline areas in medical images, which is helpful for doctors. They compare popular approaches based on U-Net, Transformer, and SAM models, explaining how each improves the accuracy and speed of segmentation. Unlike other reviews, they analyze these methods together to provide a broader perspective and highlight challenges. Their goal is to help future research and make these tools more useful in real medical settings, sharing all resources openly online.
Medical image segmentationU-NetTransformerSegment Anything Model (SAM)Evaluation metricsClinical diagnosticsDeep learningPublic datasetsImage segmentation accuracy
Authors
Pengyu Zhu, Xiaojing Zhang, Kunbo Zhang, Chunyan Zhang, Zhenyu Wang
Abstract
Medical image segmentation plays a critical role in clinical diagnostics, treatment planning, disease monitoring, and neurological disorder identification. This article presents a comprehensive review of its systematic development, covering widely used public datasets, representative methods built on the U-Net, Transformer, and SAM architectures, and key evaluation metrics with their differences, followed by an analysis of major challenges from multiple perspectives. Unlike surveys that focus on a single model family or a specific clinical application, this review organizes U-Net-, Transformer-, and SAM-based methods within a unified analytical framework, with a particular focus on their effectiveness in improving segmentation accuracy and efficiency. This work aims to guide future research and support clinical translation of medical image segmentation, with all related resources publicly available in our GitHub repository: https://github.com/andrew-pengyu/Awsome_MedSeg/tree/main.