Binary neural networks speed up real time event camera processing
Bringing BNNs to Fast Event Processing
Computer Vision and Pattern Recognition
Summary
Processing data from event cameras usually requires special methods that can handle lots of information efficiently. The authors show how binary neural networks, which use just one bit for their calculations to save resources, can be adapted to work well with this type of data. They also introduce a new way to represent event data that fits nicely with these binary networks. Their approach improves the speed and efficiency of recognizing objects from event cameras without losing much accuracy. Training the models first on regular RGB images helps improve their performance on event data.
What this means in practice
- •For embedded systems engineers: Use binary neural networks with the new binary event data format to enable faster, lower-power object recognition on event-based sensors.
- •For mobile robotics developers: Implement efficient event camera processing pipelines for real-time scene understanding with reduced computational demands.
Authors
Paul Longour, Julien Moreau, Franck Davoine
Abstract
Binary Neural Networks (BNNs) enable efficient deep learning deployment on resource constrained devices with weights and activations compressed to one bit, substantially reducing model size and inference cost. Event cameras offer complementary advantages, including low latency, high dynamic range, and low power consumption, by capturing asynchronous streams of events rather than dense image frames. Despite their shared emphasis on efficiency, the combination of these technologies remains largely unexplored. This work aims at adapting and evaluating modern deep BNN architectures on event data. We also show that cross-modal pretraining from RGB data can improve the classification accuracy of BNNs on neuromorphic datasets. We introduce the Polar-wise Binary Event Volume (PBEV), a binary representation that enables event-camera data to be processed directly by BNNs and represents a step toward fully binarized event-based vision systems. Best evaluated BNN on N-Caltech101 classification benchmarks shows 90.58% accuracy with 7.5x less operations than their full-precision counterparts.