Summary
Predicting how vehicles move is important for making controllers that drive them safely and effectively. The paper studies how well models that learn hidden representations of vehicle states keep track of physical details like position, speed, and steering angle. The authors test these models to see if they predict future vehicle behavior realistically, respond properly to small changes in commands, and keep important physical information. They find that while such models retain key details, they sometimes struggle to perfectly match the vehicle’s response to new commands, which can cause errors in control decisions.
What this means in practice
- •For autonomous vehicle engineers: Assess suitability of latent predictive models for vehicle controllers ensuring physical quantities remain interpretable and forecast responses accurately.
- •For robotic control developers: Use the measurement protocol to evaluate learned models’ fidelity for control tasks involving physical state predictions and command adjustments.
Tested on simulated data.
Abstract
Models of vehicle dynamics learned from logged states and commands complement physics-based models, and latent world models, which predict in a learned representation, are used to plan and train controllers in other domains. Vehicle controllers are usually specified in physical terms: costs, limits, and references depend on position, yaw angle, speed, and yaw rate, and the optimizer compares or differentiates predicted outcomes across nearby commands. A latent model placed in such a controller must therefore let these quantities be recovered and must change its predictions with commands as the vehicle does, and prediction error on its own latent targets measures neither. We contribute a measurement protocol for action-conditioned latent predictors with a physical readout that separately tests retention, physical-neighborhood organization, forecasting, and local response to command perturbations, using an untrained-encoder reference and three matched response paths that locate errors in the representation or the predictor. In a case study of a temporal joint-embedding predictive model trained on signals logged in IPG CarMaker, the representations retain the measured planar outputs, though an untrained encoder of the same architecture retains them slightly better; future-command input improves one-second forecasts with retention nearly unchanged; and responses to small command pulses diverge from the simulator already in latent coordinates, raising regret when choosing among nearby commands in all comparisons. Updating the predictor on responses corrects them locally at a cost in forecast accuracy. Measuring retention, forecasting, and local response separately is thus what qualifies a predictive latent as a candidate model for control, and the protocol provides the basis for its closed-loop evaluation.