Improving multi view image generation by checking geometry reliability
Self-Consistency Geometry Injection for Multi-view Consistent Image Generation
Computer Vision and Pattern RecognitionGraphics
Summary
Generating multiple consistent views of the same scene from one image is difficult because geometry predictions can be unreliable. The authors propose a method that checks how much different geometry guesses agree and uses this to decide how much to trust them when creating new views. This approach helps keep important geometric details accurate while ignoring confusing or wrong information during image generation. Their experiments show this makes multi-view images look more realistic and consistent.
What this means in practice
- •For 3d artists and animators: Generate consistent novel views from reference images while reducing errors caused by unreliable geometry predictions, improving visual coherence in 3D content creation.
- •For augmented reality developers: Create stable and coherent multi-view images that adaptively handle uncertain geometry information for better augmented reality scene rendering.
Authors
Yupeng Xiao, Yankai Wu, Jianjie Luo, Jiajie Lin, Yi Yu, Zhenguo Yang
Abstract
Recent diffusion models have achieved impressive progress in single image to multi-view generation by synthesizing consistent novel views efficiently. Existing methods usually explicitly or implicitly couple geometry inference and texture synthesis, making generation vulnerable to inaccurate geometry prediction. Although several geometry-guided methods have explored decoupling, they usually treat predicted geometry as a fixed condition and do not explicitly model whether the geometry is reliable. We therefore propose Reliability-Guided Multi-View Generation (RGMV), which comprises a parameter-free Self-Consistency Geometry Gate (SCG) and a Reliability-aware Geometry Injection (RGI) module. Specifically, a geometry proposal generator produces multiple geometry predictions for the same reference image and target viewpoints under different stochastic sampling configurations. SCG measures their spatial agreement to derive a reliability map, which RGI uses to modulate the injection strength of geometry-conditioned residuals in the RGB synthesis model. Moreover, RGMV can adapt to different geometry proposal generators. As a result, RGMV preserves stronger geometric guidance in stable regions while attenuating unreliable geometry cues before they propagate into multi-view RGB generation. Extensive experiments show that reliability-aware geometry injection improves the geometric quality of multi-view generation. Code: https://github.com/WISLab-GDUT/RGMV.