Weakly supervised AI improves polyp localization in colonoscopy videos
WSPolypNet: Weakly Supervised Polyp Localization in Colonoscopy Videos
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Detecting polyps in colonoscopy videos usually needs detailed frame-by-frame labels, which is costly and time-consuming. The authors developed WSPolypNet, a method that learns to find polyps using only video-level labels without detailed annotations. It uses a 3D neural network to highlight potential polyp areas and improves these hints with multiple viewpoints before refining them using a specialized segmentation model. This approach showed better accuracy, especially for small polyps, and could reduce the need for expensive manual labeling. Overall, their method demonstrates a practical way to assist polyp detection in videos with less detailed human input.
colonoscopypolyp localizationweak supervision3D convolutional neural networkclass activation mapsmulti-view strategysegmentation masksMedSAM2IoU thresholdrecall
Authors
Giseong Hwang, Minjae Jo, Yeonghyeon Park, Kyeonghun Kim, Seoyeon Han, Donghoon Han, Haneul Kim, Yului Jeong, Insung Hwang, Pa Hong, Ken Ying-Kai Liao, Nam-Joon Kim
Abstract
Because dense frame-level annotation of colonoscopy videos is costly, we propose WSPolypNet, a weakly supervised framework for polyp localization using only video-level labels. WSPolypNet employs a 3D convolutional neural network trained with video-level supervision to generate class activation maps (CAMs), which identify candidate polyp regions without requiring frame-level spatial annotations. The CAM-derived localization cues are further enhanced using a multi-view strategy and provided to MedSAM2 as point prompts. MedSAM2 then propagates segmentation masks across the video, refining the coarse localization cues according to polyp boundaries. WSPolypNet achieved CorLoc scores of 47.80%, 43.68%, and 35.01% at IoU thresholds of 0.3, 0.5, and 0.7, respectively, compared with 36.87%, 33.72%, and 27.94% in the single-view setting. For small polyps, the multi-view strategy improved CorLoc@0.5 from 16.01% to 30.97%. The framework also achieved a recall of 94.51%. These results demonstrate the potential of weakly supervised spatiotemporal learning to substantially reduce spatial annotation requirements for polyp localization in colonoscopy videos.