Multi-view method improves 3d shape generation with guided noise control
ReconPlusGen: Injecting Reconstruction Prior into Multi-view 3D Generation through Noise Inversion and Modulation
Computer Vision and Pattern Recognition
Summary
Creating 3D shapes from multiple pictures is a tricky job because the computer has to guess parts it can't see. The authors present a way to guide this process by predicting a rough 3D shape first and then mixing that knowledge carefully into a generative method that normally produces random results. This helps the computer keep the important parts from the pictures while still being creative to fill in missing details. Their method lets the 3D shapes look more accurate and realistic.
What this means in practice
- •For 3d content creators: Enhance 3D object models from multiple photos with improved shape accuracy and detail completion.$Commercial implications: Creates higher fidelity 3D models useful for games, VR experiences, and digital asset production.
- •For computer graphics developers: Integrate guided noise control into generative workflows to better preserve observed geometry while filling gaps.
Authors
Jiarui Liu, Heng Li, Weiyu Li, Keng Deng, Junyuan Deng, Zheng Zhongxing, Junyu Huang, Jiahao Chang, Xiaoguang Han, Ping Tan
Abstract
Qualitative results and an illustration of our core idea. Top left: reconstruction results on benchmark images. Top right: reconstruction results on real-world images. Bottom: illustration of reconstruction-guided noise initialization and modulation. Given multiple input images, we predict a point cloud in canonical space, deterministically inject the predicted geometry into the diffusion process through noise inversion, and modulate the resulting noise to preserve the generative flexibility required to complete unobserved regions and refine visible geometry.