Know-your-scene reduces errors in stereo visual slam across environments
Know-Your-Scene (KYS)-SLAM: Hierarchical Semantic-Motion Priors for Feature Matching in Stereo Visual SLAM
Computer Vision and Pattern RecognitionRobotics
Summary
Visual SLAM systems help robots and devices understand their surroundings using cameras, but they often get confused by similar-looking objects and moving things, which causes navigation errors. The authors present KYS-SLAM, which improves feature matching by using continuous scores based on the scene's context, like object types and motion, instead of just rejecting uncertain points. This method keeps more useful data for the system to work with and results in more accurate positioning across varied environments without needing to retune parameters.
What this means in practice
- •For autonomous vehicle engineers: Improve vehicle localization accuracy in dynamic urban driving scenes by using continuous semantic-motion context for feature matching.
- •For drone navigation developers: Enable more reliable indoor and outdoor drone navigation by reducing tracking errors caused by moving objects and ambiguous features.
Authors
Preeti Chatterjee, Jin Lu, Jin Sun, Suchendra M. Bhandarkar
Abstract
Stereo visual SLAM systems built on local descriptors suffer from semantic ambiguity, instance-level confusion, and independently moving objects, each corrupting data association and accumulating as trajectory drift. Prevailing semantic and dynamic SLAM methods address this through binary feature rejection, sacrificing correspondence density for outlier suppression. We contend that contextual implausibility is better expressed as a graded quantity than an exclusion criterion. We present Know-Your-Scene (KYS)-SLAM, a modular extension of ORB-SLAM3 that supplants feature rejection with continuous correspondence modulation. The contribution is the reframing of contextual evidence as correspondence cost, applied within feature matching and leaving the geometric backend unmodified. Each keypoint is augmented with semantic, panoptic, and motion priors fused through a hierarchical compatibility formulation, in which semantic class and instance identity enforce structural plausibility while a zero-shot motion score down-weights features on independently moving objects. That score comes from a training-free module fitting a depth-aware ego-motion model to background optical flow and classifying panoptic segments via self-calibrating, coverage-aware thresholds, so only segments with sufficient motion evidence are penalized and static structure is left unpenalized. Penalizing correspondences rather than discarding them preserves the geometric support bundle adjustment depends on. Under one fixed configuration, no coefficient retuned per sequence or dataset, KYS-SLAM reduces per-sequence ATE RMSE by 17.4% on outdoor KITTI and 27.7% on indoor EuRoC across 21 stereo sequences with no regressions, and by 6.6% on dynamic subsets of KITTI Tracking and 17.8%, up to 31.2%, on Virtual KITTI 2 -- cross-domain transfer across outdoor driving, indoor flight, and synthetic imagery under one set of constants.