Self-correcting method improves image and video generation models
CorrFlow: Self-Correcting Flow Matching
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Generating images and videos using AI models usually involves guessing the right motion step by step without checking past guesses. The authors introduce CorrFlow, a technique that reuses earlier calculation details to correct these guesses with a small extra step. This method improves the quality and consistency of generated images and videos without needing much additional computing power. The authors show that their approach works better than simply training the original model more and works well across different types of image and video creation tasks.
What this means in practice
- •For image generation engineers: Improve the visual quality and physical consistency of generated images by adding a lightweight correction step that reuses intermediate model information.
- •For video generation developers: Enhance text-to-video and image-to-video generation outputs using iterative self-correction without extra base model runs, boosting quality efficiently.
Authors
Hangwei Zhang, Nathan Rouillé, Chuanxia Zheng
Abstract
Flow-matching models generate samples through iterative velocity predictions, but typically do not revisit these predictions using the intermediate representations that produced them. We introduce CorrFlow, a self-correcting flow-matching framework that reuses this information through a lightweight refiner. Under rectified-flow interpolation, the error of the one-step clean prediction equals the realized velocity residual scaled by the noise level. Motivated by this identity, the refiner learns to predict residual corrections from the base model's clean prediction, conditioning, noise level and hidden states from the same forward pass. The correction is added to the base velocity before each sampling update, so subsequent predictions are made from the corrected state. This requires no additional base-model evaluations and only one lightweight refiner evaluation per step. Our theoretical analyses establish conditions under which feature reuse outperforms raw-input affine predictors and refiner training achieves greater residual-risk reduction than continued base-model training under a matched training compute budget. Experiments on class-conditional image generation, text-to-video generation and image-to-video generation show consistent improvements over the corresponding base models, including gains in visual quality and physical consistency. Project Page: https://fogradio.github.io/corrflow-project