Tracking moving objects in driver videos with camera motion removal
Referring Multi-Object Tracking in Moving-Camera Videos via Global Motion Compensation
Computer Vision and Pattern Recognition
Summary
Tracking objects in videos taken from moving cameras is tricky because the objects and camera both move. This can make things like parked cars seem like they are moving or moving cars seem still. The authors developed a way to estimate how the camera moves and then remove that motion so they can better track objects based on their actual movement. Their method improves current systems by combining this camera-motion correction with usual tracking tools.
What this means in practice
- •For autonomous vehicle developers: Track objects described by language commands in driver-view videos while correcting for camera motion to improve perception accuracy.
- •For video surveillance teams: Enhance tracking of referred moving objects from body or drone cameras by filtering out background camera motion to reduce false movement.
Authors
Hsin-Chen Pai, Jyun-Kai Wang, Yi-Cheng Peng, Wei-Ta Chu
Abstract
Referring multi-object tracking (RMOT) takes a video and a language expression as input and tracks all referred objects. Many tracking requirements involve how an object moves rather than how it appears. However, in a video captured by a moving camera, a parked vehicle may appear to move, while a moving vehicle may show little displacement. Existing RMOT methods relate motion with text but do not explicitly remove camera-induced motion. In this paper, we propose extracting residual motion across frames by estimating camera motion in driver-view videos and compare motion characteristics with the query expression. We consider the motion-matching extent and integrate it with the RMOT method's prediction result through late fusion. In the evaluation, we verify the performance gain of taking the motion compensation module as a plug-in across different RMOT hosts.