Unsupervised Pixel-Level Semantic Left-Right Understanding of In-the-Wild Images
2026-07-06 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors developed a method that helps computers figure out which parts of an image are on the left or right side at a detailed pixel level, even when only given a single photo. They combined information from 3D shapes (mostly humans and animals) with everyday images to teach the system without needing labeled data. This approach works well even for new objects like cars or trains that the system hasn't seen in 3D before. Their method performs better than previous techniques on both artificial and real images.
reflective symmetryunsupervised learning3D shape datasetssemantic segmentationleft-right predictionsingle-view imagespixel-wise predictionobject pose variationin-the-wild imagesvertex-wise understanding
Authors
Weikang Wang, Tobias Weißberg, Florian Bernard
Abstract
While various works address reflective symmetry understanding in 3D data and images, pixel-level semantic left-right prediction of in-the-wild images remains challenging, due to certain difficulties including the lack of 3D information, occlusion, object pose variation, partiality, etc. In this work, we propose an unsupervised learning framework to tackle this challenge. Leveraging recent advances in vertex-wise semantic left-right understanding of 3D data, our unsupervised learning method jointly utilises 3D shape and image datasets to infer pixel-wise semantic left-right predictions in single-view images. In particular, we show that a medium-scale 3D shape dataset comprising mainly of human- and quadruped animal-like shapes, combined with diverse in-the-wild image data, are sufficient to achieve high-quality semantic left-right prediction in images, even for entirely unseen 3D object categories, such as cars or trains. Overall, our approach achieves superior performance in dense pixel-wise semantic left-right predictions on both rendered and in-the-wild image datasets when compared to existing state-of-the-art methods.