Deep learning improves images affected by atmospheric turbulence
Deep Learning for Atmospheric Turbulence Mitigation: A Comparison Between GANs and Transformers
Computer Vision and Pattern Recognition
Summary
Atmospheric turbulence makes pictures look blurry and distorted, like looking through hot air waves. The authors compare two deep learning approaches—GANs and Transformers—to fix these distortions. They find GANs work better when images are very bad, while Transformers handle moderate to good images better, especially fixing shape warping. They also improve the Transformer method by adding a special training technique and extra image information, resulting in better overall corrections.
What this means in practice
- •For surveillance system developers: Improve real-time video quality in turbulent atmospheric conditions using tailored deep learning models for better identification and monitoring.
- •For satellite imaging engineers: Enhance satellite images distorted by atmospheric turbulence by selecting appropriate deep learning models depending on image quality levels.
Authors
Juan S. Carrillo, Jérôme Gilles
Abstract
Atmospheric turbulence creates complex spatial and temporal distortions that significantly reduce image quality. This work compares two deep learning methods for turbulence reduction: Generative Adversarial Networks (GANs) and Transformer-based models. Both methods are trained and tested on the SOTIS dataset, which includes multi-frame sequences under different turbulence conditions with ground truth references. Our results show an important finding: GAN-based models perform better in severely degraded, low-quality situations, while Transformer-based models achieve better results in moderate to high-quality settings, especially in correcting geometric distortions. We also enhance the Turbulence Mitigation Transformer (TMT) by introducing a refined training strategy, an augmented input design that includes a temporally averaged frame as prior structural information, and a modified loss function that incorporates the Structural Similarity Index (SSIM). These improvements lead to consistent performance gains across various turbulence conditions. Overall, this study provides an in-depth evaluation of deep learning methods for turbulence reduction and offers practical guidance for selecting models in different imaging environments.