Summary
Dynamic vision sensors create data events very quickly and individually for each pixel, which challenges standard computer vision methods. The authors developed ETHEREAL, a specialized chip that processes these rapid events efficiently by combining new memory and computation techniques. This design enables very fast analysis of visual data, with results available in about 25 microseconds and very low energy use. Their work helps machines use event-based cameras more effectively in real-time applications.
What this means in practice
- •For edge ai developers: Build AI systems using event-driven sensors for ultra-fast visual processing with low energy consumption at edge devices.
- •For industrial robotics teams: Integrate rapid event-based visual processing hardware to improve response times and efficiency in robot vision tasks.
Authors
Adrian Kneip, Martin Lefebvre, Daniel Gehrig, Victoria Catalán Pastor, Davide Scaramuzza, Marian Verhelst, Charlotte Frenkel
Abstract
Dynamic-vision-sensor (DVS) cameras generate events on a per-pixel basis with a $μ$s-level temporal resolution, calling for new algorithm-hardware co-design approaches compared to standard frame-based vision. While event-driven graph neural networks (EV-GNNs) emerge as a promising algorithmic solution, they raise new HW challenges by mixing dense-regular compute operations and sparse-irregular memory accesses. We present ETHEREAL, the first EV-GNN accelerator that scales to 640$\times$480 resolutions, thanks to a neighbor-parallel spline convolution engine and a 2D/3D-split memory hierarchy with a novel region-of-interest spatiotemporal caching mechanism. Measurement results demonstrate end-to-end inference with 25.6$μ$s latency and 1.7$μ$J energy per event on state-of-the-art workloads