Multi scale camera based planner improves driving trajectory prediction
S2Planner: Multi-Scale Semantic Planner for End-to-End Autonomous Driving
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Planning a safe path for autonomous cars is challenging because they need to understand where to go based on camera images and past movement. The authors developed S2Planner, which looks at images from multiple cameras at different zoom levels, along with the car’s recent movements and driving commands, to predict where the car should drive next. Instead of inventing new image or attention techniques, they combined existing ones in a way that uses the car’s position to help choose better driving paths. Their initial tests on a driving simulator showed promising results, but more testing is needed to be sure it works broadly and efficiently.
What this means in practice
- •For autonomous vehicle developers: Predict driving trajectories by combining multi scale camera images and ego motion to improve path planning in autonomous cars.
- •For robotic system engineers: Use ego conditioned trajectory initialization with iterative geometry guided image feature sampling to refine navigation waypoints in mobile robots.
Tested on simulated data.
Authors
Zhaowei Lu, Liguo Zhou, Yujie Guo, Lei Yu, Alois Knoll
Abstract
We present S2Planner, a trajectory planner that combines three front-facing cameras with ego-motion history and the current driving command. A fine-tuned DINOv3 backbone and a Spatial Tuning Adapter produce multi-scale image features; a coarse-to-fine decoder then uses trajectory self-attention and camera-projected cross-attention to refine candidate waypoints. The contribution is the integration of ego-conditioned trajectory initialization with iterative, geometry-guided sampling of multi-scale image features, rather than a new visual backbone or attention operator. On the NAVSIM v1 non-reactive evaluation, the previously reported navtest run obtained 88.03 PDMS. Because that run was selected using navtest performance, this number is exploratory and cannot be interpreted as an unbiased test estimate. Validation-selected evaluation on unexposed data, repeated runs, and computational measurements are needed to establish generalization and efficiency.