Single photon cameras offer clearer images with more measurements
One Photon, Many Worlds: Posteriors and Predictions with Single-Photon Cameras
Computer Vision and Pattern Recognition
Summary
Single-photon cameras work differently from regular cameras because their pixels only report whether they detect any photons, not how many. This creates a challenge: one snapshot can correspond to many possible images, making it hard to understand the scene. The authors show that taking more such binary snapshots reduces this uncertainty, allowing better predictions like reading QR codes or recognizing faces. They built a mathematical model that helps explain how adding more data improves the image quality and downstream tasks.
What this means in practice
- •For imaging hardware engineers: Design camera systems that adaptively collect enough photon measurements to improve low-light image reconstruction and recognition tasks using the developed probabilistic model.
- •For security system integrators: Enhance QR code scanning and facial recognition in low-light environments by optimizing the photon measurement process as described in this work.
Authors
Haejoon Lee, Mohit Gupta, Vijayakumar Bhagavatula, Aswin C. Sankaranarayanan
Abstract
Single-photon avalanche diode (SPAD) cameras operate fundamentally differently from conventional cameras due to their photon-counting nature. Each frame produces a binary image: pixels report zero if no photons arrived during exposure, and one if one or more photons arrived. Reconstructing a scene or inferring its properties from a single binary frame is difficult because many different images could produce the same measurement; thus, the inverse problem is fundamentally one-to-many. As we gather more binary measurements, the inherent uncertainty associated with the inverse problem and any associated inference diminishes. With sufficient photon counts, photon noise becomes negligible relative to the signal mean, enabling near-deterministic scene recovery and inference. This work characterizes the transition from stochastic to near-deterministic scene understanding as photon budget increases, analyzing how the stochasticity in photon arrival affects downstream inference tasks. Technically, we develop a conditional generative framework based on a Hypergeometric frame-thinning process for accumulated binary SPAD measurements. Generative models capture the one-to-many nature of photon-starved inverse problems, enabling empirical characterization of how this ambiguity diminishes with increasing measurements and its impact on downstream tasks like character recognition, QR code decoding, and facial analysis.