When Does High-CFG Diffusion Inversion Fail? A Controlled Study of Prompt--Latent Interactions
2026-07-06 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors study how well text-guided image editing works when the editing process uses different settings from the original image creation, focusing on something called classifier-free guidance (CFG). They find that some text prompts are easy to invert back to the starting point, others are hard, and some depend on the specific image details. They introduce a measure called prompt pressure to explain why some prompts are easier or harder to invert, though it doesn’t fully explain all cases. They also propose a method to improve inversion by adjusting guidance only at unstable steps, which helps make image editing more reliable.
diffusion modelsclassifier-free guidance (CFG)latent spaceinversiondenoising processtext-guided image editingprompttrajectory-consistencyPrompt-to-Prompt editing
Authors
Yan Zeng, Yusuke Hosoya, Huyen T. T. Tran, Takayuki Okatani
Abstract
Text-guided diffusion inversion is central to image editing, where an image is mapped to an initial latent and then edited by replaying the denoising process under a modified prompt. In practice, however, inversion is often performed with a lower classifier-free guidance(CFG) scale than the one used for generation or editing. This mismatch is empirically useful but leaves a basic question unresolved: when a target image is generated by a high-CFG trajectory, when can that trajectory actually be inverted? We study this question in a controlled generation--inversion--reconstruction setting, where the true initial latent and denoising trajectory are known. Using prompts taken from an existing diffusion-editing benchmark, we generate images under high CFG and reconstruct them with fixed-point inversion using the same prompt and guidance setting. The results reveal three types of prompt-level reconstruction behavior: easy prompts that reconstruct for most initial latents, hard prompts that fail for most initial latents, and intermediate prompts whose success depends on the prompt--latent pairing. To analyze the generation side, we define prompt pressure, a step-wise measure of how strongly CFG moves the denoising update away from the unconditional trajectory. Total pressure correlates with reconstruction quality and separates easy from hard prompts, but it does not explain the success or failure of intermediate prompt--latent pairs. Text-side analyses further show that the main visual subject and wording can change inversion difficulty. Finally, we evaluate a compact trajectory-consistency intervention that relaxes guidance only at locally unstable inverse steps. This diagnostic check improves reconstruction and Prompt-to-Prompt editing in our controlled setting, supporting the view that high-CFG inversion failure requires local, trajectory-aware analysis.