Real time object aware vSLAM improves navigation in dynamic scenes
CAOS-SLAM: Class-Agnostic Object-Aware Real-Time vSLAM for Dynamic Environments
Robotics
Summary
Visual SLAM systems help robots and devices understand where they are by mapping their surroundings using cameras. Moving objects like people or cars can confuse these systems, causing mistakes in mapping and positioning. The authors created CAOS-SLAM, which can detect both known and unknown moving objects without needing prior labels, using a mix of visual perception techniques. This helps the system ignore moving objects when mapping, making localization much more accurate and faster, even on regular laptops.
What this means in practice
- •For mobile robot developers: Build navigation systems that reliably localize and map in environments with unknown or moving objects without prior object models.
- •For autonomous vehicle engineers: Improve vehicle perception and motion planning by filtering dynamic objects in real-time to reduce localization errors in crowded or occluded scenes.
Authors
Gaia Borghi, João Carlos Virgolino Soares, Angelo Bratta, Shahab Heshmati-Alamdari, Claudio Semini
Abstract
Robust visual Simultaneous Localization and Mapping (vSLAM) in dynamic environments faces challenges due to the presence of moving objects that may introduce erroneous observations, corrupt the map reconstruction and overall degrade localization accuracy. Existing vSLAM approaches often rely on predefined semantic classes or costly pixel-wise classification, limiting their robustness or applicability in real-world scenarios. This paper presents CAOS-SLAM, a dynamic vSLAM framework that extends ORB-SLAM3 with a hybrid perception pipeline to detect, track, and filter both semantically known and previously unseen moving objects in real-time. On-demand panoptic segmentation is performed for object discovery, with candidate unknown objects instantiated through depth-based clustering of unclassified regions. Between panoptic updates, persistent object tracks are updated using lightweight detection when available or temporal mask propagation based on world-frame Kalman prediction, depth consistency, and appearance similarity. The estimated object motion is then used to reject dynamic features from the SLAM pipeline. Experimental results on multiple dynamic RGB-D datasets and real-world experiments demonstrate improved localization accuracy over state-of-the-art dynamic vSLAM approaches, achieving up to 59% lower localization error in scenes characterized by unknown dynamic objects and up to 88% lower error in heavily occluded scenes, while achieving a median processing rate of 30 FPS on a laptop. Furthermore, the tracking module provides estimates of object trajectories and velocities, supporting potential future applications in perception, navigation, and motion-planning tasks in dynamic environments. Code and dataset will be released upon acceptance.