Motion-consistent model improves detection and trajectory forecasting
MC-DeTra: Motion-Consistent Joint Object Detection and Socially-Aware Trajectory Forecasting in Bird's-Eye-View Images
Computer Vision and Pattern RecognitionRobotics
Summary
Predicting where cars and people will move next is important for self-driving cars. The authors worked with a prior model named DeTra that combined seeing objects and guessing their future paths but was hard to access publicly. They rebuilt DeTra and added ways to teach the model about how objects really move and how they interact with others nearby, without slowing down predictions when in use. Their improved version, MC-DeTra, better guesses the future movement of dynamic road users while keeping or improving how well it spots them in the first place.
What this means in practice
- •For autonomous vehicle engineers: Improve prediction of nearby vehicle and pedestrian trajectories while maintaining good object detection in self-driving car systems.$Commercial implications: Enables more accurate and socially-aware trajectory forecasting, critical for commercial autonomous driving products to navigate complex environments safely.
- •For urban traffic planners: Use improved models of dynamic traffic flow and social interactions to better understand and simulate road user behavior in cities.
Tested on one dataset.
Authors
Vladislav Diuzhev, Dmitry Yudin
Abstract
Unified models for object detection and trajectory forecasting aim to merge perception and prediction for autonomous driving, refining actor trajectories directly over shared bird's-eye-view (BEV) images rasterized from LiDAR and high-definition maps. Their accuracy on dynamic, moving actors, however, remains the hardest part of the task, and the strongest such model, DeTra, has no public implementation. We contribute an openly released DeTra reimplementation with documented approximations, and on top of it MC-DeTra: a family of motion-consistency mechanisms that add supervision through two annotation-derived auxiliary signals -- each actor's observed past motion and the occupancy of the surrounding traffic that forms its social context -- and one inter-output consistency constraint that aligns an actor's predicted heading with its predicted direction of motion. Every proposed loss is train-only and inference-safe: it shapes the shared BEV representation during training and is removed at test time, adding no inference latency. On the Waymo Open Dataset, evaluated under a strict, detection-conditioned forecasting protocol, MC-DeTra improves dynamic, socially-situated trajectory forecasting while preserving or improving detection accuracy; a gradient-based loss-calibration analysis exposes how the auxiliary objectives compete at the shared backbone, and our ablation identifies which signals contribute most. We release code, configurations, and evaluation tooling at https://github.com/diuzhevVlad/MC-DeTra.