Occlusion-Aware Panoptic Segmentation with Joint Position Embedding and Occlusion-Level Attention
2026-07-20 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors address the difficulty of identifying objects in crowded scenes where things block each other (occlusions). They created a tool called PEMOLA that helps computer vision models better understand occlusions by adding special attention and position information based on how much objects overlap. To test their method, they used and extended datasets with occlusion labels and found that PEMOLA improves the clarity of object detection in complex scenes without slowing down the process much. Their work shows that considering occlusions explicitly helps improve overall scene understanding.
panoptic segmentationocclusiontransformerposition embeddingattention mechanismCOCO-OLAC datasetCityscapes datasetcomputer visionocclusion-level attentionembedding modulation
Authors
Wenbo Wei, Jun Wang, Shan Raza, Abhir Bhalerao
Abstract
Panoptic segmentation in complex scenes remains challenging because of occlusions, yet modern approaches often neglect occlusion modelling. In this paper, we propose \textbf{P}osition \textbf{E}mbedding \textbf{M}odulation with \textbf{O}cclusion-\textbf{L}evel \textbf{A}ttention (PEMOLA), a novel occlusion-aware module that can be seamlessly integrated into transformer-based panoptic segmentation. To obtain occlusion cues, we train an occlusion classifier on the COCO-OLAC dataset. The classifier derives the occlusion-level attention, which serves as spatial guidance, while the occlusion labels are encoded into a learnable embedding to produce channel-wise weights. Through joint modulation, PEMOLA elegantly introduces the occlusion priors into the position embedding, thereby improving the occlusion modelling. We further annotate the Cityscapes dataset with occlusion levels, termed Cityscapes Occlusion Labels for All Computer Vision Tasks (Cityscapes-OLAC), following the same labelling protocol as COCO-OLAC, to evaluate the cross-dataset generalisation ability of PEMOLA. Extensive experiments on COCO-OLAC and Cityscapes-OLAC demonstrate that PEMOLA consistently improves panoptic segmentation quality while introducing minimal computational overhead. These results highlight the importance of occlusion modelling, where incorporating occlusion-level attention helps deliver robust panoptic segmentation under occlusion. Code and dataset are available at https://github.com/wenbo-wei/PEMOLA.