Ultratex speeds up high-resolution 3d texture generation by dropping background data
UltraTex: Unleashing 2K Multi-View Diffusion for 3D Texturing
Computer Vision and Pattern Recognition
Summary
Creating detailed surface colors for 3D objects needs high-quality textures, but existing methods struggle with very high resolution images because they use too much memory and take too long. The authors show that by ignoring parts of the image that aren't important (background) and focusing only on the object itself, their system called UltraTex can generate much sharper textures faster. They also built a huge dataset to train their method to work well at 2048 resolution. UltraTex speeds up training and inference many times while capturing fine details well.
What this means in practice
- •For game developers: Generate high-detail textures for 3D game assets more efficiently, enabling richer visuals without prohibitive computing costs.$Commercial implications: This enables game studios to produce and sell more realistic 3D models with less compute, improving content production pipelines.
- •For 3d content creators: Produce high-resolution textured models for animation or virtual production faster by removing unnecessary background calculations.
Authors
Yibo Zhang, Ze Yuan, Nan Cao, Li Zhang, Yan-Pei Cao, Yuan-Chen Guo, Rui Ma
Abstract
High-quality texture generation is essential for creating realistic and production-ready 3D assets. Recent multi-view diffusion methods have shown promising results for image-guided 3D texturing, but they are typically constrained to low operating resolutions such as 512 or 768, making it difficult to preserve high-frequency details from high-resolution reference images. Scaling this paradigm to 2048 resolution is computationally prohibitive, as the unified multi-view sequence exceeds 212K tokens and incurs excessive memory and latency. In this paper, we present UltraTex, an efficient end-to-end framework for high-resolution multi-view diffusion-based 3D texturing. Our key observation is that object-centric multi-view renderings contain two major sources of redundancy: background-induced sequence redundancy and sparse token interactions within the foreground. To address them, we introduce Background Token Dropping, which removes background tokens before the DiT backbone, and Block-Sparse Attention, which reduces attention computation over the retained foreground sequence. To enable efficient foreground-only inference while avoiding reconstruction artifacts, we further design Foreground-Aware VAE Decoding to ensure the quality of the final high-resolution views. To satisfy the demanding data requirements of 2K-resolution multi-view diffusion training, we construct G-buffer TexVerse, a large-scale, ultra-high-resolution multi-view rendering dataset covering over 268,000 3D assets. Extensive experiments show that UltraTex generates visually faithful textures with rich fine-grained details, while substantially improving efficiency, achieving $20.6\times$--$91.1\times$ training speedup and $22.3\times$--$74.6\times$ end-to-end inference speedup over the baseline on common samples in our dataset. Code and data is at https://yiboz2001.github.io/UltraTex.