Domain adaptive panoptic segmentation becomes simpler stronger and more robust
MC-PanDA++: Simpler, Stronger, and More Robust Domain-Adaptive Panoptic Segmentation
Computer Vision and Pattern Recognition
Summary
Panoptic segmentation helps computers understand images by labeling every pixel. But it’s hard to teach computers when the training images are different from the ones they see in the real world. The authors improved a method that helps computers learn from easy-to-label images (like synthetic ones) and adapt to new real-world pictures without extra labels. Their new approach uses better starting models, adjusts training automatically for different classes, and simplifies the training process, making the segmentation more accurate and stable across different types of images.
What this means in practice
- •For autonomous vehicle developers: Improve scene understanding in self-driving cars by adapting segmentation models trained on synthetic data to real-world road images without extra labeling.$Commercial implications: Enables better perception models for autonomous vehicles that generalize to diverse driving conditions with less labeled data.
- •For industrial robot programmers: Develop robots that recognize and segment objects accurately in new factory environments using models adapted from simulation data without manual annotation.
Authors
Ivan Martinović, Josip Šarić, Yuki M. Asano, Siniša Šegvić
Abstract
Unsupervised domain adaptation (UDA) reduces the annotation burden in panoptic segmentation by leveraging a cost-effectively labeled source domain (e.g., synthetic) and an unlabeled target domain to bridge the distribution gap. Existing panoptic UDA methods rely on teacher-student consistency learning built upon suboptimal per-pixel segmentation architectures. In contrast, state-of-the-art mask transformers are rarely adopted due to their pronounced vulnerability to confirmation bias in consistency learning, where erroneous teacher predictions are reinforced during training. Our earlier approach, MC-PanDA, mitigates this issue through fine-grained confidence estimation, which suppresses gradients from unreliable masks while sampling informative yet reliable locations for loss computation. However, this method entails a complex multi-stage training and requires careful hyperparameter tuning. This work presents MC-PanDA++, which addresses these limitations by introducing: (i) self-supervised vision encoders that provide a stronger and more robust initialization, further reducing the reliance on human annotations, (ii) per-class, self-adapting mask-wide loss scaling that stabilizes training and enables the usage of a single set of hyperparameters across domains, and (iii) a single-stage training pipeline that decreases overall conceptual complexity. Together, these improvements result in a conceptually simpler, better-performing, and more robust method for domain-adaptive panoptics. Source code: https://github.com/martinovicivan/MC-PanDA