OmniFabric improves photorealistic 3D garment textures from single images
OmniFabric: Coherent UV Space Texture Synthesis for 3D Garment Reconstruction
Computer Vision and Pattern Recognition
Summary
Creating detailed 3D clothes from just one photo is hard because making realistic colors and patterns is tricky. The authors introduce OmniFabric, a new way to create these textures clearly and accurately on a flat sewing pattern. They use smart AI tools to first guess a rough texture, then a special machine learning model cleans and fixes it to look like real fabric, without unwanted shadows or distortions. This helps make 3D clothes that can be used in animations or games more easily and realistically.
What this means in practice
- •For 3d artists and animators: Automatically generate clean, realistic garment textures for 3D models from a single image to speed up asset creation.
- •For game developers: Create detailed and physically accurate 3D clothing textures that support lighting changes and simulation on game characters.
Authors
Ding-Jiun Huang, Yuanhao Wang, Cheng Zhang, Hugo Bertiche, Alexandru-Eugen Ichim, Thabo Beeler, Fernando De la Torre
Abstract
Automated generation of production-ready 3D garment assets from a single image is a central challenge in digital content creation. While recent generative models have significantly advanced 3D geometry reconstruction, synthesizing high-quality textures remains a bottleneck. Existing methods often bake environmental illumination and shadows directly into the texture map, or they fail to maintain global structural coherence, making the resulting assets unusable for physical simulation and relighting. In this work, we introduce OmniFabric, a novel approach that synthesizes globally coherent texture maps directly within the 2D sewing pattern space. Given a single reference image, our pipeline utilizes an estimated 3D mesh and generative priors of powerful Vision-Language Models (VLM) to establish a complete but coarse texture initialization across the unwrapped sewing patterns. We then leverage a specialized diffusion transformer, trained via an automated synthetic data engine and conditioned on 3D positional features, to refine this initialization directly in the canonical UV domain. This effectively removes distortion and baked-in artifacts to extract a clean and normalized texture map that preserves the original garment design. Extensive experiments demonstrate that OmniFabric significantly outperforms state-of-the-art baselines, yielding photorealistic 3D garments with high-quality textures.