High-resolution depth guides privacy-safe segmentation with low-res color
Privacy-Preserving Semantic Segmentation from High-Resolution Depth and Ultra-Low-Resolution RGB
Computer Vision and Pattern RecognitionRobotics
Summary
Cameras on mobile robots can invade people's privacy by showing detailed images. This paper looks at a new way to keep important geometric details using high-resolution depth sensors while reducing privacy risks by capturing ultra-low-resolution color images. The authors created a method that combines these two inputs to identify objects and regions in a scene accurately, even with less color detail. Their approach works well for both 2D images and 3D spaces, and it makes it harder to recover sensitive visual information. They also tested the method on real robots navigating environments.
What this means in practice
- •For robotics engineers: Enable mobile robots to map and understand spaces while reducing visual privacy risks using combined high-res depth and low-res RGB data.
- •For security system designers: Implement room monitoring systems that retain spatial understanding but obscure identities by using asymmetric sensing of detailed depth and blurry color images.
Authors
Xuying Huang, Swithinraj Moses Daniel, Sicong Pan, Sebastian Houben, Maren Bennewitz
Abstract
As mobile robots become increasingly integrated into everyday environments, privacy risks arising from onboard cameras have become a growing concern. Ultra-low-resolution (ULR) RGB can mitigate visual privacy exposure at the source, but ULR appearance alone substantially limits semantic and spatial understanding. We therefore introduce a privacy-preserving asymmetric sensing setting that combines high-resolution (HR) depth with ULR RGB, preserving dense geometry while restricting fine-grained visual information. To address the severe information imbalance between HR depth and ULR RGB, we propose a joint 2D framework using HR geometry to guide semantic-oriented RGB reconstruction and RGB-D segmentation. Despite reliable frame-level predictions, consistent scene-level understanding remains challenging under the asymmetric HR depth--ULR RGB setting. We therefore develop an end-to-end 2D-to-3D pipeline that consolidates 2D semantic features for 3D segmentation. Experiments on ScanNet show that our method achieves the best 2D and 3D segmentation performance among privacy-preserving approaches and delivers the strongest zero-shot transfer to SUN RGB-D and SceneNN. Privacy recoverability analysis shows that our proposed HR depth--ULR RGB input reduces the recoverability of sensitive data, and real-robot experiments demonstrate the utility of the resulting 3D semantics for object-goal navigation.