Neuromorphic motion detector enables fast pinball ball tracking
Can Spiking Neural Networks play pinball? A neuromorphic motion detector for target tracking
Computer Vision and Pattern Recognition
Summary
Tracking a fast-moving ball in real time is hard while saving energy. The authors created a system that mimics how the brain processes visual motion with spiking signals, using a special event-based camera and a brain-like computer chip. Their setup senses the ball's position and speed, then reacts very quickly to hit it with virtual pinball flippers. The system performs better than humans in tests and works both in simulation and with a physical machine without needing to learn from experience.
What this means in practice
- •For robotics engineers: Build fast, low-energy vision systems for robot motion detection and target tracking using spiking neural networks and event-based sensors.
- •For embedded systems designers: Implement energy-efficient neuromorphic perception-to-action pipelines for real-time control in constrained hardware environments.
Authors
Mazdak Fatahi, Šárka Pryjmaková, Pierre Boulet, Giulia D'Angelo
Abstract
Biological visual systems achieve continuous, low-latency motion perception by processing sparse, asynchronous spiking signals, enabling real-time tracking under strict energy constraints. Event-based cameras, inspired by the mammalian retina, replicate this efficiency by capturing only local brightness changes as asynchronous events, offering a natural substrate for spiking neural networks (SNNs) to parallelise computation and adapt to fast-changing scenes. Pinball provides a controlled yet dynamic testbed, requiring precise motion estimation and fast reaction to a small, rapidly moving target. This work presents a fully spiking, real-time perception-to-action pipeline for closed-loop pinball gameplay. A dynamic vision sensor observes a small, fast-moving ball, and a network of spiking Time-Difference Encoders on the SpiNNaker neuromorphic platform jointly estimates its position, speed, and direction. The system is characterised across receptive field size, accumulation window, and angular tuning width for real-time operation, and benchmarked in closed loop against human players across two flipper regimes of increasing physical realism. It achieves a hit rate of 56.1%, nearly double the human average, reacting within 21.7 ms (5 ms network latency) and consuming an estimated 148 μW using fewer than 25k neurons, among the fastest and most energy-efficient event-based closed-loop demonstrators benchmarked. Under more realistic flipper dynamics, tuning a single interpretable policy parameter reproduces the full spectrum of human play styles, from cautious to aggressive, with no change to the perception pipeline. A physical demonstrator, tracking a real ball and actuating real flippers in closed loop, confirms the principle operates beyond simulation. Its fully spiking, learning-free design offers a compact, energy-efficient example of real-time neuromorphic perception-to-action.