SlackDrive improves real-time driving model efficiency using runtime latency
SlackDrive: Reclaiming Runtime Slack for Adaptive Driving Inference
Artificial IntelligenceRobotics
Summary
Self-driving cars use complex models to predict the future and help plan driving actions, but these models can be slow and sometimes take too long to run. The authors noticed that the available computation time can change depending on what is happening right now. So, they created SlackDrive, a system that watches the actual time taken by the car’s computer during each step and adjusts how much computing the model uses next time to stay within the required speed. This method keeps the car’s decision-making quick and accurate, improving performance without breaking time limits.
What this means in practice
- •For autonomous vehicle engineers: Optimize onboard model inference to maintain strict real-time control while improving prediction accuracy under changing compute loads.
- •For real-time embedded system developers: Use runtime latency feedback to adapt compute budgets dynamically, enhancing performance within latency constraints in systems with shared onboard resources.
Authors
Xiaohuan Pei, Hengguang Zhou, Yuanhao Ban, Justin Cui, Jiaqi Feng, Haoyu Xie, Tao Huang, Pichao Wang, Yanchao Yang, Cho-Jui Hsieh
Abstract
Driving world-action models improve planning by coupling multimodal reasoning with future prediction, but their growing inference cost increasingly conflicts with the real-time latency requirements of vehicle control. Existing acceleration methods reduce tokens, layers, or sampling steps with policies selected prior to deployment, yet leave residual runtime variation largely unexploited after offline profiling and static scheduling on shared onboard compute. We observe that the largest admissible compute budget varies systematically with the residual runtime state, while recent realized latency provides a direct signal of the available compute slack. Motivated by this observation, we propose \textbf{SlackDrive}, a pre-inference compute allocator that reuses realized latency to select the compute budget of each control step before model execution. SlackDrive profiles the latency and planning utility of a small discrete budget set once, estimates online compute state from completed forwards, and selects the highest-utility budget predicted to remain within the admissible latency envelope, complementing existing profiling and resource scheduling while preserving the driving backbone and its compute actuator. On NAVSIM v2 with DriveDreamer-Policy, SlackDrive improves latency-constrained EPDMS by $21.7\%$ over the strongest baseline under a stringent latency regime, while the full-budget model and preconfigured token-pruning baselines exceed the admissible latency envelope under runtime contention.