Gate3DGen improves 3D image quality by selectively cleaning photos
Gate3DGen: A Pre-processing Model that Selectively Purifies Real-World Images for 3D Generation
Computer Vision and Pattern Recognition
Summary
Creating 3D models from a single photo works well with high-quality images but struggles with real-world photos that might be blurry, dark, or partially blocked. The authors designed Gate3DGen, a tool that first checks if a photo needs cleaning and only fixes it when necessary, keeping important details for 3D modeling. They trained this system using synthetic images with known problems, so it learns what needs fixing. Tests show this method quickly improves the quality of 3D models from challenging photos while preserving what makes objects recognizable.
What this means in practice
- •For 3d content creators: Enhance the quality of 3D models generated from imperfect real-world photos without changing existing 3D generation tools.
- •For augmented reality developers: Improve input images' clarity selectively to ensure better 3D asset generation for AR applications in varied lighting and occlusion conditions.
Authors
Xiaolin Li, Zihao Huang, Fanpeng Meng, Chen Yang, Guanjun Wu, Kai Zhang, Jiemin Fang, Qi Tian
Abstract
Single-image 3D generation has achieved high fidelity on curated benchmarks, yet degrades sharply on real-world images with uneven illumination, blur, and occlusions. Purifying such inputs requires deciding when to intervene and preserving the visual cues needed for 3D generation, while leaving clean inputs unchanged. We introduce Gate3DGen, a selective pre-processing model that gates image inputs to existing 3D generators without retraining them. A degradation-aware router bypasses inputs classified as clean and extracts features to guide purification of the remaining inputs. Conditioned on these features, a 3D-oriented one-step purifier purifies degraded inputs with spatial losses that encourage preservation of object identity, contours, and textural cues. To provide paired supervision, we construct a synthetic data engine that renders clean views of 3D assets and synthesises composite degradations. On the synthetic benchmark, the purifier improves image fidelity and downstream 3D geometry in only 2.3s on average, reducing TRELLIS.2 mesh distance by 61.5% relative to degraded inputs. VLM evaluation and human preferences on real photographs further support identity preservation and degradation correction. More demos and code are available at https://supra-3d.github.io/gate3dgen.