Video super-resolution speeds up with streaming-aware diffusion model
Streaming-Aware Diffusion for Real-Time Video Super-Resolution via Cross-Step Attention
Computer Vision and Pattern Recognition
Summary
Making videos clearer and sharper in real time is hard because it needs a lot of computing quickly. The authors present a way to improve this by adapting existing image-enhancing AI models to work better on videos. Their method shares information between video frames during the refining steps without heavy extra processing. This approach helps produce smoother, more natural-looking videos at fast playback speeds.
What this means in practice
- •For video streaming engineers: Enhance live streaming video quality in real time without added delay or hardware upgrades.
- •For mobile app developers: Integrate real-time video enhancement features into mobile apps with efficient computation on limited hardware.
Authors
Harris Partaourides, Sotirios Chatzis
Abstract
Real-time video super-resolution requires high spatio-temporal fidelity under strict latency constraints, challenging diffusion models due to their iterative sampling cost and limited temporal coordination. We propose a streaming-aware framework that adapts pretrained single-image latent diffusion models for efficient video super-resolution (VSR) by exploiting the sequential structure of video streams. Our Cross-Step Attention mechanism reuses intermediate denoising features across adjacent frames and diffusion steps, enabling temporal information exchange without explicit temporal modeling. We further introduce Trajectory-Coupled Diffusion Scheduling, which aligns adjacent diffusion states and provides cleaner intermediate representations for cross-step conditioning, improving temporal coherence. These components are integrated into a streaming inference pipeline that incrementally propagates latent states across frames, reducing the effective computational complexity from $O(N \cdot S)$ to $O(N + S)$ for $N$ frames and $S$ diffusion steps. Experiments on REDS4 and YouHQ40-Test demonstrate improved perceptual quality and temporal realism while maintaining frame-wise stability. Our method achieves over 40 FPS at $512 \times 512$ resolution after cold start, enabling real-time VSR without explicit temporal modeling.