Semi supervised segmentation improves accuracy with dual branch voting
HDVS: semi-supervised semantic segmentation via heterogeneous dual-branch voting supervision
Computer Vision and Pattern Recognition
Summary
Semantic segmentation means dividing images into meaningful parts, but it usually needs many labeled images which can be expensive. This paper presents a new way to do segmentation by using lots of unlabeled images plus a few labeled ones, improving the process with two different models that 'vote' to decide the best labels. This voting helps avoid mistakes that can happen when models learn from each other's errors. They also add a feature-level step to make the two models different yet consistent, helping the final result. Tests on standard image sets show this method works better than previous ones.
What this means in practice
- •For autonomous vehicle developers: Improve object and scene understanding by training segmentation models with less labeled data using dual-branch voting for more reliable predictions.
- •For medical image analysts: Enhance segmentation of medical images where labeled data is scarce by using semi-supervised dual-branch voting supervision to reduce annotation needs.
Authors
Yongqi Shan, Yunzhi Zhuge, Huchuan Lu
Abstract
Semi-supervised semantic segmentation, which aims to utilize large volumes of unlabeled images to achieve accurate segmentation results with fewer human annotations, has attracted increasing attention. Prior methods were primarily based on the assumption that the pseudo-labels generated by each branch were sufficiently reliable to supervise the remaining branches. However, there is a concern that the errors in pseudo-labels could accumulate during co-training, potentially resulting in suboptimal performance. To this end, we present a novel framework called heterogeneous dual-branch voting supervision (HDVS), which is designed to enhance the reliability of pseudo-labels and mitigate the issues arising from pseudo-labeling. Specifically, based on the cross-supervision framework, we introduce a voting mechanism that correlates pseudo-labels from heterogeneous branches to produce pseudo-labels with enhanced reliability. Concurrently, we employ a feature communication module to introduce perturbations at the feature level in each branch to maximize prediction diversity across the dual branches while maintaining convergence. Comprehensive evaluations of the proposed HDVS on two benchmark datasets (PASCAL VOC 2012 and Cityscapes) demonstrate its superiority to the state-of-the-art approaches.