Random filter losses improve image reconstruction without tuning
RaFL: An Empirical Study of Random Filter Losses for Image Reconstruction
Computer Vision and Pattern RecognitionMachine Learning
Summary
Image reconstruction programs often lose fine details when they only focus on matching pixels exactly. Using features from pretrained networks can help, but those features are usually made for specific types of images. The authors study a method that compares images using random filters instead, which works across different image types without needing special tuning. This method is fast to run and performs as well or better than standard pixel-based methods on various 2D and 3D image tasks.
What this means in practice
- •For medical image processing teams: Use random filter losses for stable and tuning-free 3D medical image reconstruction improving detail preservation over pixel losses.
- •For computer vision engineers: Incorporate RaFL to develop image reconstruction models that need no manual loss tuning and handle diverse 2D input domains efficiently.
Authors
Ruibo Wang, Fengbei Liu, Aleksei Tiulpin, Mert R. Sabuncu, Heejong Kim
Abstract
Pixel losses train image reconstruction networks toward a point statistic of the target distribution, often averaging away fine textures under uncertainty. Perceptual losses alleviate this issue using pretrained features, but those features are inherently domain-specific. We study a random filter loss (RaFL) that compares prediction and target through filters with random weights, avoiding both pretrained features and task-specific loss engineering. RaFL combines random convolutions, random Fourier features, and random wavelet filters, automatically balancing their contributions through gradient-norm matching, eliminating manual loss-weight tuning and grid search. We systematically investigate three design choices: filter family, weight resampling schedule, and weight initialization. Based on validation PSNR and stability across random seeds, we select the configuration that uses all three filter families, redraws filters every optimization step, and employs Gaussian initialization. We evaluate RaFL on three 2D and two 3D image reconstruction tasks. RaFL outperforms a pixel loss on three tasks and matches it on two, and demonstrates stable performance across random seeds on all five tasks. RaFL adds less than 2% to the backbone's forward FLOPs and no inference cost, requiring $182\times$ fewer loss FLOPs than a VGG19 perceptual loss. These findings suggest that RaFL is a simple, tuning-free, and domain-agnostic loss objective for 2D and 3D image reconstruction.