Afforddriv3d improves self-driving by predicting safe driving areas
AffordDrive3D: Affordance-Aware World-Action Modeling with Spatial Understanding
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Driving a car safely requires understanding both the layout of the road and which parts are safe to drive on. Most current models predict future scenes mainly based on camera images, missing key details about where the car can actually go or where it might crash. The authors created AffordDrive3D, a model that not only estimates the future geometry of the driving scene but also identifies safe and risky areas for the car. This combined prediction helps self-driving cars plan better routes and avoid collisions.
What this means in practice
- •For autonomous vehicle engineers: Plan car trajectories using future predictions of safe and risky driving areas along with 3D scene structure to improve driving safety.
- •For robotic navigation developers: Improve navigation by combining spatial layouts with action-relevant region forecasts for safer robot movement in complex environments.
Authors
Tianhui Cai, Xinglong Sun, Chao Fang, Zhenxin Li, Rui Song, Jose M. Alvarez, Yunxiang Mao, Jiaqi Ma, Langechuan Liu
Abstract
World-action models have recently improved autonomous driving by jointly learning future scene prediction and trajectory generation. Most existing approaches model the future primarily through RGB appearance, and recent works have begun to incorporate geometric prediction to improve spatial understanding. However, dense geometry describes the spatial layout of the entire scene without indicating which parts are most relevant to the ego vehicle's action. For driving, the model must also identify and anticipate where it can safely move and which regions may pose collision risks. Jointly modeling action-relevant regions and future geometry can provide the policy with both driving-relevant cues and their corresponding spatial structure. We therefore propose AffordDrive3D, an affordance- and geometry-aware world-action model that jointly learns future action-relevant regions and spatial structure. In order to capture the scene semantics and driving context needed for driving affordance prediction, we build AffordDrive3D on a VLM backbone to forecast drivable areas and collision-critical regions that directly affect ego motion, while predicting future geometry from RGB world-model latents. On NAVSIM, AffordDrive3D achieves state-of-the-art performance with 91.3 PDMS and 89.9 EPDMS, demonstrating the effectiveness of jointly modeling future affordances and geometry for trajectory planning.