GeoWAM: Visual Geometry World Action Models for Autonomous Driving
2026-08-24 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionRobotics
AI summaryⓘ
The authors propose GeoWAM, a model for autonomous driving that predicts how a scene changes by focusing on 3D shapes (point clouds) rather than 2D images. They argue that using 3D geometry better captures the actual layout and movement of objects, making it easier to understand and predict driving actions. GeoWAM learns to forecast the future geometry of the scene and then uses this to plan the car's future path. Tests show that this approach leads to better driving decisions than methods relying on image predictions alone.
World action modelsAutonomous drivingPoint cloudsScene geometryEgo trajectoryVisual representation3D scene understandingSpatial structureScene evolutionPretraining objectives
Authors
Yiren Lu, Xin Ye, Jiaming Liu, Jin Yao, Yi-chung Chen, Liam Merino, Dhruva Dixith Kurra, Min Cai, Tom Lampo, Yu Yin, Danhua Guo, Burhan Yaman
Abstract
World action models (WAMs) have recently gained increasing attention as a framework for jointly modeling scene evolution and ego actions in autonomous driving. Most existing WAMs learn scene dynamics in pixel space by combining a video-generation backbone for future-observation prediction with an action head for ego-trajectory prediction. Pixels, however, provide only an indirect representation of these dynamics: they entangle geometry and motion with appearance, texture, and illumination, forcing the model to infer three-dimensional transformations from two-dimensional observations. We argue that geometry, represented by point clouds, offers a more natural state space for driving because it explicitly captures spatial structure and the rigid and non-rigid transformations that govern scene evolution while directly aligning with the space in which driving actions are executed. Building on this insight, we introduce \textbf{GeoWAM}, a visual geometry world action model for autonomous driving. Rather than predicting future images, GeoWAM is pretrained to forecast future scene geometry, yielding representations that jointly encode spatial structure and temporal evolution. A geometry-conditioned action head then leverages these learned geometric dynamics to predict future ego trajectories. Extensive open-loop and closed-loop evaluations show that visual geometry world modeling yields substantially stronger driving policies than image-based alternatives, establishing future-geometry prediction as an effective pretraining objective for autonomous driving.