Event-to-Video Reconstruction using Spatio-Temporal and Frequency-Enhanced Deep Neural Networks
2026-05-25 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors developed a new method called MSFET-E2V to turn data from event cameras into video frames. Unlike previous methods that only look at spatial information, their approach also uses frequency information to better capture fine details and reduce visual errors. They achieve this by combining spatial and frequency features with a special attention module and a lightweight wavelet-enhanced skip block. Their experiments show that this method produces clearer videos from event data while being faster and requiring less memory than existing transformer-based methods.
event cameraevent-to-video reconstructiontransformer modelspatial domainfrequency domaindiscrete wavelet transformcross-domain attentionskip connectionartifact suppressionspatio-temporal features
Authors
Ramna Maqsood, Paulo Nunes, Luís Ducla Soares, Caroline Conti
Abstract
Event cameras offer significant advantages over conventional frame-based counterparts, including high temporal resolution, low latency, and energy efficiency. These characteristics make them suitable for high-speed and high-dynamic range scene acquisition scenarios; however, the lack of dense intensity frames limits the direct applicability of conventional computer vision methods for scene understanding. Event-to-video (E2V) reconstruction seeks to bridge this gap by converting asynchronous event streams into a sequence of synchronous video frames. Existing E2V reconstruction methods based on convolutional neural networks and transformers operate primarily in the spatial domain and often struggle to recover fine structural details while suppressing severe reconstruction artifacts. To address these issues, we propose MSFET-E2V, a novel multiscale frequency-enhanced transformer model. At its core lies a cross-domain attention module, which fuses spatio-temporal features with frequency-aware representations derived from the discrete wavelet transform. Unlike prior methods relying solely on spatial attention, our approach effectively captures both local and global structures by taking into account low- and high-frequency components, enhancing detail preservation and robustness across various motion scenarios. Furthermore, we propose a lightweight wavelet-enhanced skip block that serves as a skip connection, facilitating artifact suppression and structural detail refinement through joint spatial-frequency domain processing. Extensive experiments demonstrate that MSFET-E2V achieves superior performance over state-of-the-art methods on multiple real-world event datasets, offering significant gains in reconstruction quality. Moreover, compared to the existing transformer-based method, our proposed model significantly reduces the number of parameters, the GPU memory usage, and inference time.