High frame rate HDR video created from single low dynamic frame and events
HIVE-Diff: High Frame Rate HDR Video Reconstruction from a Single LDR Frame and Events
Computer Vision and Pattern Recognition
Summary
Capturing fast-moving scenes with both bright and dark areas is hard because normal cameras struggle to get clear images at high speed and wide lighting ranges. The authors developed a new method called HIVE-Diff that uses a single normal (low dynamic range) frame combined with event camera data, which records quick changes in brightness, to create multiple high-quality high dynamic range video frames. They trained their system mostly on simulated data and then adapted it to real-world footage. This approach helps produce sharp, colorful videos in challenging lighting and motion conditions.
What this means in practice
- •For autonomous vehicle developers: Generate high frame rate HDR video for navigating environments with rapid movement and varied lighting using a single standard camera frame plus events.$Commercial implications: Enables improved vision systems in self-driving cars with enhanced video clarity and speed from limited sensor inputs.
- •For scientific imaging teams: Produce high-quality HDR video capturing fast events and extreme lighting using fewer frames and event data for efficient experimental analysis.
Authors
Caixin Wang, Qingrui Wang, Jie Zhang, Ralph Etienne-Cummings
Abstract
High frame rate HDR video is important in applications such as autonomous driving and scientific imaging, where rapid motion and extreme illumination often occur simultaneously. Conventional frame cameras face a trade-off between temporal resolution and dynamic range, while event cameras provide complementary high-speed brightness-change measurements over a wide illumination range. These properties have motivated event-based HDR reconstruction, with existing methods broadly following event-only and hybrid event-frame paradigms. Event-only HDR reconstruction, however, suffers from brightness and color drift due to the lack of absolute intensity references. Hybrid event-frame methods alleviate this ambiguity using low-dynamic-range (LDR) frames as anchors, but most perform one-to-one HDR reconstruction at available LDR timestamps. In this work, we investigate single-reference HDR temporal expansion, reconstructing N HDR frames from one LDR reference and events instead of N aligned LDR observations. We develop HIVE-Diff, an event-conditioned video diffusion framework that decomposes the task into event-guided temporal generation and HDR content completion. Because paired real HDR-event data are scarce, we train on synthetic data and adapt the model to real event domains using LDR-event sequences as proxy supervision with teacher-student distillation. Experiments on synthetic and multiple real-world datasets demonstrate strong reconstruction quality and robust real-data adaptation.