ESAR: Event-Based Synthetic Aperture Reconstruction
Computer Vision and Pattern Recognition
Summary
The authors study how to create images from event cameras, which capture changes in light instead of normal pictures. They treat the problem like looking at a fixed scene through a moving camera, connecting how the scene changes over time to the event data. They find a way to invert this process mathematically to recover a simpler representation of the scene’s brightness. Their method works well on simulated and real data, producing stable large-scale images while reducing small noisy details, compared to other existing methods.
Authors
Harbir Antil, Daniel Blauvelt, David Sayre
Abstract
Event cameras report asynchronous polarity events when changes in log--radiance exceed a fixed contrast threshold, producing signed temporal contrast measurements rather than conventional image frames. We formulate monocular event-based imaging as a synthetic-aperture inverse problem for a static ground-domain log--radiance field $θ\in \mathbb{R}^{N_g}$. Instead of reconstructing a latent pixel-time volume $v \in \mathbb{R}^{N_pN_t}$, we impose the geometric relation $v=Pθ$, where $P$ maps the fixed scene into motion-dependent latent views. Aggregating events over finite time intervals gives the linearized model \[ APθ= b+η, \] where $A$ is a temporal differencing operator, $b$ contains signed binned event counts, and $η$ represents measurement and modeling errors. This decomposition exposes a synthetic-aperture structure: under near-nadir motion, successive projections are approximately shifted views of a common scene, while the composite operator $AP$ remains ill-conditioned because it combines spatial averaging with temporal differencing. We therefore use regularized inversion to recover $θ$. Numerical experiments on simulated data and real near-nadir Falcon Neuro event data show that the proposed $θ$-based formulation recovers coherent large-scale spatial structure, relative to dynamic latent-image and learned event-reconstruction baselines, while suppressing fine-scale texture.