Fast video correction removes wide-angle lens distortions for stable full-body view
Distortion-Aware Spatiotemporal Learning for Real-Time Wide-Angle Video Rectification
Computer Vision and Pattern Recognition
Summary
Wide-angle camera lenses show a lot of distortion, especially near the edges, making people look stretched or unnatural. Existing ways to fix this focus mostly on faces and can't handle videos well at speed, causing shaky results. The authors created FastVC, a new method that fixes these distortions smoothly across full-body views in videos and runs fast enough for real-time use. They combined physics-based mapping with smart attention to keep images stable over time and also made a new dataset to test wide-angle video corrections.
What this means in practice
- •For video app developers: Create apps that correct wide-angle video distortions in real time for smoother full-body videos.$Commercial implications: Enables commercial video editing and streaming apps to offer fast, high-quality wide-angle distortion fixes previously unavailable.
- •For augmented reality developers: Improve AR user experiences by stabilizing wide-angle video input from cameras with minimal delay and distortion.
Authors
Lang Nie, Yihan Yang, Wenbo Nie, Hongguang Zhu, Yang Wei, Chunyu Lin, Bin Xiao
Abstract
While wide-angle lenses provide an expanded field of view (FoV), they exhibit severe nonlinear distortion near image boundaries, unnaturally stretching people in these regions. Existing methods mostly focus on facial regions, require high computational costs, and lack temporal modeling, resulting in incomplete full-body rectification, impractical real-time deployment, and noticeable inter-frame shakes. To tackle these issues, we propose \textbf{FastVC}, an efficient full-body video rectification framework driven by distortion-aware spatiotemporal learning. Grounded in lens physics, FastVC introduces a distortion-aware thin-plate spline (DA-TPS) module that nonlinearly remaps control points into a peripheral-dense layout, enabling more accurate geometric fitting without additional computational cost. We further develop a dual-level spatiotemporal learning scheme for causal and stable video rectification. At the model level, neighborhood spatiotemporal attention (NST) aggregates local context and selectively fuses historical features with low overhead. At the optimization level, sparse spatiotemporal supervision applies spatial alignment only to anchor frames, decoupling frame-wise accuracy from temporal stability. Besides, we contribute the first benchmark dataset for wide-angle full-body video correction. Extensive experiments show that FastVC achieves state-of-the-art rectification fidelity and visual stability at 57.0 FPS.