Local learning improves training of spiking neural networks for event vision
Toward On-Chip Training of Spiking Neural Networks for Dense Event-Based Vision
Computer Vision and Pattern Recognition
Summary
Training spiking neural networks (SNNs) to interpret the fast, asynchronous signals from event cameras is hard because it needs much computer memory. The authors present DELL, a method that trains these networks in smaller parts using local feedback, making memory use much lower while still getting better results. They tested DELL on tasks like estimating motion and segmenting images and found it beats previous local methods and even full training in some ways. This approach could help robots and devices process event-based vision more efficiently.
What this means in practice
- •For robotics engineers: Train spiking neural networks for fast event-based vision with less memory consumption on embedded systems.
- •For embedded systems developers: Implement efficient local learning methods for dense vision tasks on neuromorphic or low-resource chips.
Authors
Maxime Vaillant, Axel Carlier, Lai Xing Ng, Christophe Hurter, Benoit R. Cottereau
Abstract
Event cameras provide low-latency, asynchronous visual sensing for resource-constrained robotics. Spiking neural networks (SNNs) process event streams naturally, but training deep SNNs with backpropagation through time (BPTT) requires substantial memory and remains difficult on neuromorphic hardware. Local learning avoids this by restricting error propagation to local blocks, but existing methods mainly target classification rather than dense prediction. We introduce DELL (Dense Event-driven Local Learning), a block wise scheme for dense event-based vision that replaces global gradient propagation with local dense supervision. Learnable, spatially structured local heads supervise each block at its appropriate resolution while preserving temporal dynamics within blocks. We evaluate DELL on optical-flow regression and semantic segmentation with a fully spiking U-shaped architecture. On DSEC optical flow, DELL reduces peak training memory by 39.6% relative to end-to-end BPTT while improving accuracy, reaching 1.670 px endpoint error on the official test benchmark versus 1.941 px for the same backbone trained end-to-end. Block detachment behaves more like a regularizer than a constraint. DECOLLE, the existing local-learning baseline, relies on fixed random local read-outs poorly suited to dense regression, resulting in a 3.9x higher endpoint error; learnable local heads recover this loss and outperform end-to-end training across all optical-flow metrics. On segmentation, they recover most of the performance gap, although DELL remains a few mIoU points behind end-to-end training. With 2.3M parameters, 24x fewer than the strongest SNN baseline, the backbone remains competitive with the SNN state of the art on DSEC. These results extend local learning to dense event-based prediction while substantially reducing training memory.