Summary
End-to-end driving models often struggle when actually controlling a vehicle because they are trained by copying driving paths without checking if every part of the path can be smoothly followed in real time. The authors noticed that while final destinations predicted by these models are usually accurate, the intermediate points in the path can be unrealistic or hard to drive through. They introduced a method called Endpoint-Constrained Optimization (ECO) that adjusts the intermediate points to make the path easier to track, without changing the final destination. This method improves driving performance in simulations without needing extra maps or retraining the model.
What this means in practice
- •For autonomous vehicle developers: Improve driving control of end-to-end models by refining intermediate waypoints to enable smoother and more feasible trajectories without retraining or additional maps.
- •For robotics simulation engineers: Increase accuracy of vehicle simulations by applying a lightweight correction layer that adjusts trajectory points to better reflect achievable vehicle motion.
Authors
Brayden Zhang, Mahsa Golchoubian, Igor Gilitschenski, Boris Ivanovic, Kashyap Chitta
Abstract
End-to-end driving policies are commonly trained through open-loop behavior cloning, yet they must ultimately operate in closed-loop when deployed on a vehicle, creating a fundamental mismatch between training and execution. Beyond the commonly studied effects of covariate shift and causal confusion, we identify a complementary factor for this open-loop/closed-loop gap: waypoint-based supervision and displacement metrics do not ensure that the intermediate trajectory is physically coherent or easy for the controller to track. We observe that these inconsistencies concentrate primarily at intermediate waypoints, while the predicted endpoint remains comparatively reliable. Based on this observation, we introduce Endpoint-Constrained Optimization (ECO), a lightweight postprocessing layer that anchors the trajectory to the vehicle's executed history, preserves the policy's predicted endpoint, and reshapes the intermediate waypoints to improve feasibility. ECO requires no map, privileged simulator state, or additional training, and can be inserted between a broad range of waypoint-emitting policies and their controllers. Across two closed-loop simulators, it improves the aggregate closed-loop score of all six evaluated generative and regression-based driving policies, and the gains tend to increase with how often the base plans violate motion limits. On HUGSIM, ECO improves VaVAM from 18.1 to 31.0 HD-Score (+71%), achieving 1st place on the HUGSIM Closed-Loop Driving Challenge. Similarly, on AlpaSim, ECO increases the scene scores of VaVAM and DiffusionDrive by 123% and 22%, respectively. These results show that for a broad collection of end-to-end driving models, repairing the intermediate geometry of predicted trajectories without changing the policy's predicted endpoint can substantially improve closed-loop performance.