Stereo cameras can be tricked to misjudge obstacle distances
Illusion of Depth: Revealing Hidden Stereo Vision Vulnerabilities in Depth Estimation
Cryptography and Security
Summary
Stereo cameras estimate how far away things are, which helps self-driving cars and robots avoid crashes. This paper shows that attackers can fool these cameras by placing simple repeating patterns, causing the cameras to wrongly judge distances by meters. Deep learning methods used for depth estimation are also vulnerable. The authors tested their findings on popular cameras and driving systems, showing even quick attacks can trigger emergency brakes. They also tried defense methods and proposed a better way to detect and reduce these errors.
What this means in practice
- •For autonomous vehicle teams: Identify and defend against subtle stereo camera attacks that falsely shift obstacle distances, ensuring safer self-driving car operations.
- •For robotics developers: Enhance depth perception reliability by detecting and suppressing depth errors caused by repeating patterns in stereo vision systems.
Authors
Sri Hrushikesh Varma Bhupathiraju, Tetsu Ishizue, Nicholas U. Costagliola, Ozora Sako, Kentaro Yoshioka, Takeshi Sugawara, Sara Rampazzi
Abstract
Stereo cameras are integrated into autonomous systems such as self-driving cars, drones, and robots to offer precise depth estimation in a cost-effective manner compared to LiDAR technology. In this work, we reveal an intrinsic vulnerability in stereo cameras that stems from their pixel sampling and calibration processes, which can influence the outputs of stereo matching algorithms. Attackers can achieve fine-grained control over the estimated depth of real obstacles using simple repeating patterns, without relying on sophisticated adversarial machine learning techniques. Furthermore, deep learning-based depth estimation models exhibit a similar vulnerability. We evaluate the impact of this attack on two widely used stereo matching algorithms (BM and SGBM), three deep learning models (PSMNet, MoCha-Stereo, and UniMatch), a stereo-LiDAR fusion model (SGM-DDC), and two popular commercial stereo cameras, the ZED2 and Intel RealSense D435. For example, in the ZED2 camera, an attacker can displace obstacles up to 20~meters farther or 12~meters closer. In our real-world evaluation in a driving setting, a brief 0.5~second attack can trigger emergency braking in a popular autonomous driving framework. We further demonstrate the feasibility at driving speeds up to 40~km/h using CARLA. Finally, we confirm the ineffectiveness of state-of-the-art defenses, and we propose a novel strategy that leverages similarity scores to dynamically detect and suppress the depth discrepancies. Our work highlights vulnerabilities hidden in stereo matching and deep learning depth estimation models, addressing critical limitations in autonomous system deployments.