Taotex improves texture details for accurate 3d material generation
TaoTex: Boosting Texture Detail Fidelity for Native 3D Material Generation
Computer Vision and Pattern Recognition
Summary
Creating 3D objects with detailed textures like text and patterns is hard for current models. The authors introduce TaoTex, a new method that uses better training data and smart ways to combine big-picture and tiny details. They also improve how the model learns fine texture details and handle multiple viewpoints for consistent results. This helps produce 3D materials with clearer, more accurate textures than before.
What this means in practice
- •For 3d artists and designers: Generate 3D materials with clearer and more precise textures for use in games, films, and virtual environments.$Commercial implications: Enables production of high-quality textured 3D assets for creative industries requiring realistic visual details.
- •For augmented reality developers: Create consistent and detailed 3D material textures viewable from multiple angles to enhance AR visual fidelity.
Authors
Xiuchao Wu, Shuichang Lai, Jiangjing Lyu, Chengfei Lyu
Abstract
Recent 3D generation models can produce accurate geometries while still struggling to reconstruct detailed textures. We propose a diffusion-based native 3D material generation model TaoTex, which faithfully recovers intricate textures through tailored strategies and improvements. First, we develop a data construction agent to create high-frequency textured 3D assets to bridge the data gap in public datasets. Training with these data significantly enhances the ability of TaoTex to recover challenging details such as text and patterns. Second, we design a multi-level feature fusion (MLFF) module to adaptively integrate local and global features of the conditional input, providing more complete texture cues for the diffusion model and thereby enhancing reconstruction fidelity. To alleviate VAE reconstruction errors, we adopt a latent-to-pixel space loss transition, further improving the pixel-level details and generation quality. Finally, we scale TaoTex to multi-view inputs by incorporating learnable viewpoint embeddings, achieving accurate and consistent material reconstruction across views. Extensive experiments demonstrate that our method significantly outperforms existing approaches in preserving texture details in both single- and multi-view settings.