Vitality-Aware Compression for Efficient Image-to-Shape Diffusion Transformers
2026-07-01 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors created a new way to make image-to-shape Diffusion Transformer models smaller without losing the quality of the 3D shapes they generate. These models are usually very large and hard to run on devices with limited computing power. Their approach focuses on which parts of the model are most important for creating the shapes and reduces size by pruning and adjusting the model intelligently. They were able to shrink models by up to two-thirds while keeping the 3D output just as good. This method can be easily used with different 3D shape generation models to make them more efficient.
Diffusion Transformers3D shape generationmodel compressionstructured pruningquantizationfine-tuningimage-to-shapegeometric fidelityresource-constrained settingsbackbone compression
Authors
Jaeah Lee, Hyunjin Kim, Jaewoong Cho, Gihyun Kwon
Abstract
We propose the first compression approach for image-to-shape Diffusion Transformers (DiTs) that substantially reduces model size while preserving geometric fidelity. Despite remarkable progress in 3D shape generation, large DiT-based models remain computationally prohibitive in resource-constrained settings. Furthermore, it is difficult to directly transfer existing diffusion model compression strategies developed for different domains to 3D generation, and prior 3D efficiency approaches focus primarily on inference speed rather than backbone compression. To address this limitation, we build a geometry-aware compression framework tailored to image-to-shape DiTs. Guided by the observation that 3D DiT layers exhibit non-uniform importance for geometry synthesis, we introduce a vitality-guided framework integrating structured pruning, adaptive quantization, and targeted fine-tuning. Our method achieves up to 66% model-size reduction across state-of-the-art image-to-3D models while maintaining synthesis fidelity comparable to full-sized counterparts. This highlights the potential of our framework as a plug-and-play solution for efficient 3D shape generation across diverse models.