Visual autoregressive priors improve raw to sRGB color and detail recovery
Visual Autoregressive Priors for RAW-to-sRGB Image Signal Processing
Computer Vision and Pattern RecognitionMultimedia
Summary
Converting raw camera sensor data into colorful, detailed images is tricky because of missing or imperfect camera settings. The authors use a visual autoregressive model, which predicts image details in a step-by-step way based on learned patterns from many images. They also introduce a new way to measure color and detail quality in different frequency bands. This approach makes images look closer to real photos by better preserving structure, though perfect color matching is still difficult. Tests on a standard dataset show improved image quality compared to previous methods.
What this means in practice
- •For camera firmware engineers: Enhance raw-to-sRGB pipelines to yield more faithful colors and finer detail for smartphone and digital cameras using autoregressive image priors.
- •For mobile photo app developers: Improve photo color accuracy and structural detail in apps that convert raw images to display-ready sRGB for users under varying conditions.
Tested on one dataset.
Authors
Tailai Chen, Xiaotong Luo, Yuan Gao, Xin Jin, Wenjun Zeng
Abstract
RAW-to-sRGB image signal processing (ISP) must recover perceptually faithful colors and fine details from sensor measurements, often under imperfect spatial alignment and missing camera metadata. This paper presents, to the best of our knowledge, the first application of visual autoregressive (VAR) next-scale prediction over a discrete image codebook to the RAW-to-sRGB ISP task. We adapt a frozen 1.10\,B-parameter VAR backbone for RAW-conditioned ISP with only 32.93\,M trainable parameters (2.99\%), and propose a frequency-decomposed color loss that separately supervises low-frequency tone via wavelet LL cosine similarity and chromatic edges via detail-band $\ell_1$. On the Zurich RAW-to-sRGB benchmark, the method improves PSNR-Y from 21.31 to 21.89\,dB and reduces LPIPS from 0.276 to 0.218 on the full 1,204-image test set. Diagnostic experiments show that the VAR prior preserves structure well, but continuous color transfer remains the dominant bottleneck: oracle affine correction recovers 3.8\,dB, while learned color heads yield marginal gains.