Quanta Perception as Probabilistic Events
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
The authors introduce a new way for robots to understand the world using sensors that count individual photons, called quanta sensors. Instead of relying on traditional methods that combine photons over time and struggle in low light or fast movement, their approach uses a Bayesian method to quickly update beliefs about changes in light intensity. This allows the system to generate useful signals like motion and uncertainty in real time, even in very dim conditions, without needing to retrain vision models. Their method works much faster than current techniques and can run on common hardware, improving robotic perception where photons are scarce or scenes change quickly.
quanta sensorsphoton countingBayesian inferenceevent camerasreal-time perceptionlow-light visionrecursive belief statespose estimationdynamic rangecomputational primitives
Authors
Varun Sundar, Pavan Thodima, Sacha Jungerman, Mohit Gupta
Abstract
Autonomous systems rely on extracting information from light, yet remain brittle in extreme environments, from nighttime navigation to high-speed robotics. Conventional sensors aggregate photons over fixed exposures, imposing trade-offs between sensitivity, dynamic range, and temporal resolution that degrade perception when photons are scarce or dynamics are rapid. Quanta sensors detect individual photons, but their streams exceed real-time compute and latency budgets by orders of magnitude. Here we introduce $\textit{probabilistic events}$, a computational primitive for real-time quanta perception from individual photon detections. By computing the posterior over the time since the last intensity change, we represent photon streams as recursive belief states. Rather than fixed-threshold event-camera triggers, this recursive Bayesian formulation yields three low-latency signals: motion-adaptive scene flux, high-fidelity activity maps, and entropy-based perceptual uncertainty. This representation enables perception in extreme conditions, including pose estimation of a running person at $\sim$0.05 lux---without retraining vision models. Our approach processes input streams exceeding 50{,}000 quanta frames per second on commodity GPU hardware---yielding kilohertz-scale outputs up to four orders of magnitude faster than state-of-the-art quanta reconstruction baselines, even for megapixel arrays. By replacing frame reconstruction with direct probabilistic inference over photon streams, this work bridges photon-counting quanta sensing with robotic vision.