Spike-driven planning matches neural network accuracy with low energy
SDPAD: A Fully Spike-Driven Pipeline for End-to-End Autonomous Driving
RoboticsArtificial Intelligence
Summary
Planning safe routes for self-driving cars usually needs powerful computers that use a lot of energy. The authors built a method called SDPAD that uses spiking neural networks, which work more like the brain by sending brief signals, to plan driving paths with much less energy. They showed that SDPAD can plan as accurately as common neural networks while using less than 2% of the energy. This could help make self-driving technology more efficient, especially on small devices.
What this means in practice
- •For autonomous vehicle engineers: Run energy-efficient real-time trajectory planning on low-power onboard hardware without losing accuracy.
- •For embedded systems developers: Integrate spike-based neural processing pipelines to reduce energy consumption in complex perception and planning tasks.
Authors
Chengjun Zhang, Yuhao Zhang, Jie Yang, Mohamad Sawan
Abstract
End-to-end autonomous driving demands trajectory planners that are both highly accurate and cheap enough for edge deployment. State-of-the-art artificial neural network (ANN) planners meet the accuracy requirement at the cost of heavy dense computation, while spiking neural networks (SNNs)---though promising orders-of-magnitude energy savings through sparse, event-driven arithmetic---still lag far behind in planning accuracy. We present \textbf{SDPAD}, a fully spike-driven end-to-end planning pipeline that closes this gap. SDPAD converts a pre-trained ANN perception stack into integer-spike form via quantized ANN2SNN conversion, lifts multi-view images into the bird's-eye-view (BEV) space with a spike-driven-max (SDM) depth distribution (Spike-3D-Lift), and plans through the Spike-QFormer, a spiking query transformer in which ego, agent, and map queries distilled from the BEV scene are fused by learnable waypoint queries via cross-attention, followed by deformable spike-cross-attention refinement. Every operation is gated by integer spikes and inference is a single feed-forward pass without temporal simulation loops. On the nuScenes open-loop benchmark, SDPAD achieves an average $L_2$ error of 0.40\,m and a collision rate of 0.12\%, on par with strong ANN planners while consuming 69.9\,mJ---less than 2\% of recent ANN baselines. In closed-loop evaluation on the NAVSIM navtest split, SDPAD reaches 86.3 PDMS, surpassing the previous SNN planner SAD by 4.3 points and matching mainstream ANN planners at a fraction of their energy. To our knowledge, SDPAD is the first fully spike-driven planner evaluated in end-to-end autonomous driving, demonstrating that SNNs can rival dense ANNs in complex driving tasks.