UMI3D: Robust 3D Generation on Unconstrained Multi-Image Inputs via Simultaneous Focus Cross-Attention Routing
2026-07-27 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors found that 3D models trained to create objects from one image struggle when given multiple images, often making messy or wrong results. They discovered this happens because the model doesn’t know which image to trust for each tiny part of the 3D object. To fix this, they built UMI3D, a tool that helps the model focus on the best image for each part during the creation process without extra training. This approach improved 3D generation from multiple images using existing single-image models.
3D foundation modelscross-attentionmulti-image inputvoxeldenoising stepimage routingSimultaneous Focus Cross-Attention (SFC-Attn)Voxel Reference Score (VRS)3D generation
Authors
Zefan Qu, Zhenwei Wang, Gerhard Petrus Hancke, Rynson W. H. Lau
Abstract
Recent 3D foundation models can generate high-quality assets from a single image, but degrade markedly on unconstrained multi-image inputs, often producing distorted geometry, over-smoothed textures, and chaotic colors. We argue that this failure stems not from limited model capacity, but from a mismatch between single-image cross-attention and the multi-image setting: existing models lack a principled way to decide which image each 3D voxel should trust at each denoising step. Revisiting recent single-image 3D foundation models, we show that explicitly routing each voxel to its most informative image is sufficient to unlock strong performance on inconsistent multi-image inputs. Based on this observation, we propose UMI3D, a training-free and plug-and-play framework that restructures cross-attention for unconstrained multi-image 3D generation. Its core, Simultaneous Focus Cross-Attention (SFC-Attn), activates all conditioning images at each denoising step while allowing each voxel to focus on the single image that best explains it. To enable this routing, we derive the Voxel Reference Score (VRS), a model-intrinsic metric for voxel--image affinity that requires no external matching, segmentation, or correspondence models. Extensive experiments show that UMI3D unlocks the multi-image potential of single-image 3D generation frameworks across diverse tasks. Project Page: UMI3D-Project.github.io.