AI summaryⓘ
The authors address the challenge of detecting local image quality issues, especially when models trained on fake (synthetic) distortions struggle with real-world pictures, called the synthetic-to-authentic problem. They created a large dataset called VIGIL-140K with many carefully distorted images to train their model, VIGIL, which uses multiple detection layers from a large language model to catch different kinds of distortions simultaneously. Their approach also keeps track of subtle distortion signs even when images appear normal, helping with tricky cases where foreground and background are hard to separate. Tests showed their model works better than existing methods both on fake distortions and real-world examples.
Image Quality Assessment (IQA)Large Multimodal Models (LMMs)Synthetic-to-Authentic (S2A) ProblemDistortion DetectionSynthetic DistortionsForeground-Background SeparationMulti-level Feature DetectionSupervised Fine-Tuning (SFT)Visual Distortion Injection
Authors
Ziheng Jia, Yingji Liang, Jiaying Qian, Xiongkuo Min
Abstract
The localized depiction of perceptual quality has long been a crucial, yet underexplored, challenge in image quality assessment (IQA). Existing approaches based on large multimodal models (LMMs) predominantly rely on text-driven supervised fine-tuning (SFT). However, this training paradigm exhibits notable limitations in detection accuracy. Moreover, synthetically distorted images, which are often used as the primary training data source, show a significant generalization gap when deployed in real-world scenarios; thus, the \textbf{synthetic-to-authentic (\textit{S2A})} problem represents a critical challenge. Motivated by these issues, we propose \textbf{\textit{VIGIL}}, which leverages the LMM architecture for precise visual distortion detection. From a candidate pool of over 1000K samples, we construct the \textbf{\textit{VIGIL-140K}} training set, which consists of over 140K distorted images. These images are obtained through rigorous quality filtering and carefully crafted distortion injection, covering 8 major synthetic distortion categories. Our model leverages different layers of the large language model (LLM) decoder, treating them as \textit{multiple detectors} that perform synchronous distortion detection using multi-level features. Additionally, we retain distortion cues from predictions assigned to the non-distortion class, which helps mitigate the ambiguous foreground-background (\textit{FG-BG}) separation commonly encountered in the \textit{S2A} problem. After post-processing, our model consistently outperforms strong baselines on both in-domain synthetic distortion detection and \textit{S2A} tasks.