Segment moving objects in video despite camera motion without training

Segmentation of Moving Objects During Camera Motion Using the OWL Function

Computer Vision and Pattern RecognitionRobotics

Summary

When a camera moves, it can be hard to tell which parts of the scene are objects moving on their own versus just shifting because of the camera. The authors use special visual clues about how objects look like they grow or rotate to figure out the direction each object is moving. By grouping parts of the image that move the same way, they can split the scene into separate moving objects. This method works without needing prior knowledge about the scene, depth measurements, or any learning from examples.

What this means in practice

Authors

Daniel Raviv, Juan D. Yepes

Abstract

We present a principled, analytical framework for segmenting multiple independently moving rigid objects during single camera motion, built directly upon the OWL perception function. OWL encodes two fundamental visual motion cues, perceived looming $L$ and perceived rotation $ω$, into a single closed-form representation of relative 3D structure. A key consequence of this structure is that the unit direction of relative translation $\hat{t}$ is analytically recoverable per image point from $L$ and $ω$ alone. All points belonging to a non-rotating rigid body share the same $\hat{t}$, whereas points on different rigid bodies generally yield distinct values. This furnishes a natural segmentation criterion: clustering per-point $\hat{t}$ estimates partitions the visible scene into independently moving rigid bodies. Building on the 2D complex form of OWL, we derive the framework in 3D using quaternions, for a single moving camera. The approach is parallel, camera rotation-invariant, and requires no training data, no depth information, and no semantic priors.