DriveReferee uses geometry rules for safer driving model decisions
DriveReferee: Geometric Safety Verdicts Need Not Be Learned for Driving World-Action Models
Computer Vision and Pattern Recognition
Summary
Safe self-driving cars need to check if their planned paths will cause crashes or go off the road. The authors show that instead of teaching a computer to guess if a path is safe, it’s better to teach the computer to figure out where things are and then use clear safety rules based on geometry. Their system, DriveReferee, predicts the scene from a camera view and applies explicit safety checks. This approach works as well or better than previous methods that learned safety indirectly, needing a lot less data and no extra training for the safety checks.
What this means in practice
- •For autonomous vehicle engineers: Use DriveReferee to verify self-driving car trajectories for collisions and road safety using only camera input with no extra state data.
- •For robotics system designers: Incorporate geometric safety checks in robots that navigate dynamic environments without relying on learned safety verdicts.
Authors
Fengcheng Yu, Dhruv Parikh, Junjie Ye, Maulik Bhatt, Thang Vu, Igor Vasiljevic, Vitor Guizilini, Yue Wang
Abstract
Generative world-action models (WAMs) jointly generate future video and vehicle actions, while their action branches remain primarily optimized by expert imitation. Yet imitation provides no explicit closed-loop geometric verdict for generated trajectories, making verification important during both training and deployment. Closed-loop evaluators can check collision and drivable-area violations, but require privileged scene state unavailable at deployment. Existing approaches often close this gap by learning a verifier from sensor features. For these geometric checks, the rule itself is explicit. For example, collision is determined by whether the rolled-out ego footprint overlaps occupied vehicle space. What is unavailable at deployment is the scene state needed to apply the rule. We introduce DriveReferee, which uses a learned geometry readout to predict the scene representation from camera observations and executes the geometric safety rule directly rather than learning it. The resulting analytic referee evaluates collision and drivable-area safety from a scene state and candidate trajectory. During training, it scores self-sampled trajectories on ground-truth state and distills the resulting preferences into the WAM policy. At deployment, the same referee evaluates generated trajectories on this predicted state and selects a safer alternative when needed. The analytic referee requires no verdict-specific training, and its decisions follow an explicit geometric rule. Under matched candidates and inference budgets, it matches or outperforms all learned-verifier and heuristic baselines. Given the same predicted state and trajectory, learning the verdict provides no measurable downstream gain despite requiring tens of thousands of evaluator-labeled training examples. On the full NAVSIM navtest, DriveReferee reaches 92.02 PDMS with single-camera visual input and no external training data.