FactorDrive: Adaptive Multi-Step Reasoning Driven by Planning-Critical Factors for End-to-End Autonomous Driving
2026-08-10 • Robotics
RoboticsComputer Vision and Pattern Recognition
AI summaryⓘ
The authors present FactorDrive, a new system for self-driving cars that improves how the car plans its next moves by paying closer attention to important factors in the environment. They created a specialized dataset, PCF-CoT, to help the model reason step-by-step using physical and spatial clues specific to each driving scene. They also developed a method called QS-GRPO that uses smart search techniques to find better planning paths and then teaches the system to follow these improved plans. Tests on standard driving benchmarks show their approach leads to better driving decisions. Overall, the authors address key gaps in how current models plan safe and efficient driving routes.
Vision-language modelsAutonomous drivingPlanning reasoningChain-of-thought (CoT)Spatial-physical evidenceMonte Carlo Tree Search (MCTS)Policy optimizationTrajectory planningInstruction tuningClosed-loop benchmarking
Authors
Guolei Huang, Tengfei She, Yuxuan Lu, Yao Huang, Yuqi Ye, Yongjun Shen
Abstract
Vision-language models (VLMs) have advanced scene understanding and enabled explicit reasoning in end-to-end autonomous driving. However, existing methods insufficiently integrate spatial-physical evidence into planning reasoning, while reasoning adaptation remains coarse-grained and falls short of scene-specific planning demands. Furthermore, reasoning-path optimization for higher planning quality remains largely unexplored in autonomous-driving post-training. To address these limitations, we propose FactorDrive, an end-to-end autonomous driving framework for adaptive multi-step reasoning driven by planning-critical factors (PCFs). We first perform large-scale driving-domain instruction tuning to establish foundational driving knowledge. Building on this foundation, we construct PCF-CoT, a chain-of-thought (CoT) dataset that grounds planning reasoning in trajectory-relevant spatial-physical evidence and organizes reasoning around scene-specific PCFs, enabling the composition and depth of reasoning paths to adapt to different planning demands. We further introduce Quality Search-Guided Group Relative Policy Optimization (QS-GRPO), which guides Monte Carlo Tree Search (MCTS) with trajectory-level planning rewards to discover reasoning paths with higher planning quality and uses the resulting responses to optimize the policy through GRPO, thereby improving trajectory planning performance. Extensive experiments on both open-loop (nuScenes) and closed-loop-oriented (NAVSIM) benchmarks demonstrate that FactorDrive achieves state-of-the-art planning performance.