Segment moving objects in video despite camera motion without training
Segmentation of Moving Objects During Camera Motion Using the OWL Function
Computer Vision and Pattern RecognitionRobotics
Summary
When a camera moves, it can be hard to tell which parts of the scene are objects moving on their own versus just shifting because of the camera. The authors use special visual clues about how objects look like they grow or rotate to figure out the direction each object is moving. By grouping parts of the image that move the same way, they can split the scene into separate moving objects. This method works without needing prior knowledge about the scene, depth measurements, or any learning from examples.
What this means in practice
- •For computer vision engineers: Segment independently moving rigid objects in video streams without requiring depth data or training sets.
- •For drone navigation teams: Detect and separate moving obstacles from background during drone flight using only onboard monocular cameras.
Authors
Daniel Raviv, Juan D. Yepes
Abstract
We present a principled, analytical framework for segmenting multiple independently moving rigid objects during single camera motion, built directly upon the OWL perception function. OWL encodes two fundamental visual motion cues, perceived looming $L$ and perceived rotation $ω$, into a single closed-form representation of relative 3D structure. A key consequence of this structure is that the unit direction of relative translation $\hat{t}$ is analytically recoverable per image point from $L$ and $ω$ alone. All points belonging to a non-rotating rigid body share the same $\hat{t}$, whereas points on different rigid bodies generally yield distinct values. This furnishes a natural segmentation criterion: clustering per-point $\hat{t}$ estimates partitions the visible scene into independently moving rigid bodies. Building on the 2D complex form of OWL, we derive the framework in 3D using quaternions, for a single moving camera. The approach is parallel, camera rotation-invariant, and requires no training data, no depth information, and no semantic priors.