Contextrast++: Robust Multi-Scale Contextual Contrastive Learning for Semantic Segmentation
2026-08-24 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionRobotics
AI summaryⓘ
The authors present Contextrast++, a method to improve semantic segmentation by better understanding both local details and the bigger picture in images. Their approach includes a special learning technique that combines features at different scales and uses smart sampling of difficult image areas, especially along object boundaries. They also address the problem of some classes appearing less often by balancing feature representation through a memory system. Tests show their method improves segmentation accuracy without slowing down the final prediction process. Overall, this helps computers label images more precisely.
semantic segmentationcontrastive learningmulti-scale featuresclass imbalanceadaptive fusionpixel-to-anchor lossanchor-to-anchor lossmemory bankboundary-aware samplingfeature representation
Authors
Changki Sung, Hyungtae Lim, Wanhee Kim, Youngwoo Seo, Hyun Myung
Abstract
Semantic segmentation has rapidly advanced with deep learning; however, challenges remain in effectively capturing local and global contexts as well as addressing the long-tailed distribution problem. To tackle these issues, we present Contextrast++, a robust contrastive learning method for semantic segmentation that improves multi-scale feature integration and mitigates class imbalance issues. Our method consists of two key components: 1) contextual contrastive learning (CCL) and 2) boundary-aware negative (BANE) sampling. CCL includes three subcomponents: adaptive fusion module, pixel-to-anchor (PA) loss, and anchor-to-anchor (AA) loss. The adaptive fusion module dynamically balances local and global feature integration, resulting in a more context-aware representation. While the PA loss leverages the fused multi-scale features to improve feature representation learning, the AA loss focuses on addressing the long-tailed distribution problem by utilizing a memory bank that stores a fixed number of class-balanced representative anchors. Meanwhile, BANE sampling enhances segmentation precision by selecting hard negatives from misclassified boundary regions, which refines fine-grained details during contrastive learning. As verified in extensive experiments using public datasets, we demonstrate that Contextrast++ substantially improves semantic segmentation performance over existing contrastive learning-based state-of-the-art approaches, while introducing no additional computational overhead during inference.