Dynamic object detection improves by mixing motion and depth cues
DynOSeg: Dynamic Object Segmentation via Motion-Geometry Fusion
Computer Vision and Pattern Recognition
Summary
Identifying moving objects in videos is tricky because both the camera and objects can move, causing confusing changes in what we see. The paper presents DynOSeg, a new way to spot moving things by first combining information about how the scene's shape and movement work together, then adding knowledge about what objects are to better separate moving objects from the background. The authors also created a new synthetic driving dataset to help train their method. DynOSeg performs better than previous methods on multiple tests.
What this means in practice
- •For autonomous vehicle developers: Enhance detection of moving vehicles and pedestrians by fusing motion and depth information for safer navigation in complex scenes.$Commercial implications: Makes more accurate dynamic object detection modules for self-driving cars commercially viable by improving segmentation quality in urban environments.
- •For video surveillance teams: Improve automatic identification of moving people or objects in security footage by combining visual motion data with depth and semantic understanding.
Authors
Xu Han, Yiduo Wang, Yiran Wang, Viorela Ila
Abstract
Moving Object Segmentation (MOS) detects and separates dynamic objects from the static background in a video, which plays a fundamental role in dynamic-scene understanding. Most MOS methods rely on pure visual motion cues alone, such as optical flow or point tracks, which is inherently ambiguous due to the compounded camera and object motions as well as motion-induced appearance changes, e.g. shadows and reflections. Naive fusion of additional information such as depth and semantics, however, cannot leverage their complementarity to the fullest extent. We therefore propose DynOSeg, a novel MOS approach that takes on a two-stage fusion structure. DynOSeg first combines a geometry-aware residual map and depth encoding into a geometric representation of motion, delineating camera motions and object dynamics. We then further incorporate semantic features to refine object understanding. We have additionally curated a diverse training corpus including CARLA-MOS, a new synthetic urban-driving dataset for MOS training. Our experiments show that DynOSeg consistently outperforms published state-of-the-art MOS methods across a diverse range of datasets. Our code and pretrained models can be found at https://github.com/ACFR-RPG/DynOSeg