Silsa improves 3d shape generation with fewer tokens and better structure
SILSA: Sliding-Window Slice Latents for Topology-Preserving High-Resolution 3D Generation
Computer Vision and Pattern RecognitionArtificial IntelligenceMachine Learning
Summary
Generating detailed 3D shapes usually involves breaking them into many tiny pieces, which can be slow and can mess up delicate connections. The authors created SILSA, a new way to represent 3D shapes using overlapping slices that keep surfaces continuous and connected. SILSA uses fewer tokens, making generation faster and more memory efficient, while better preserving thin parts and overall shape connections. Their tests show SILSA achieves higher accuracy and better shape quality compared to previous methods.
What this means in practice
- •For 3d graphics developers: Generate high-resolution 3D models with better connectivity and less computation for games and simulations.
- •For medical imaging software teams: Improve reconstruction of complex anatomical structures with fewer errors and better preservation of thin or connected shapes.
Authors
Tianjiao Yu, Xinzhuo Li, Yifan Shen, Ying Shen, Kiet A. Nguyen, Adheesh Sunil Juvekar, Ismini Lourentzou
Abstract
High-resolution 3D generation increasingly relies on voxel latents and multi-stage pipelines that first predict active structure and then synthesize local geometry. While effective, this design fragments continuous surfaces into many local tokens, inflates generation cost, and often weakens topological consistency for thin or highly connected shapes. We introduce SILSA, a topology-aware 3D generation framework that represents shapes with compact sliding-window slice latents. Instead of generating expensive voxel tokens, SILSA uses a fixed set of overlapping slices along the three canonical axes, where each token summarizes a local depth window to preserve cross-sectional continuity and support single-stage rectified-flow generation. A Slice VAE encodes oriented surface samples into multi-axis slice latents and reconstructs them with a sparse volumetric decoder, while a Volumetric Anchor Lattice coordinates directional slice streams through a shared 3D workspace. To preserve structural correctness, we introduce slice-level topology supervision that matches persistence diagrams and aligns Betti transitions across neighboring slices. Experiments show that SILSA improves structural fidelity while substantially reducing generation cost. SILSA improves PSNR by $8.7\%$, coverage by $5.96$ absolute points, and Betti error by $9.2\%$ over the strongest baseline, while using $70.0\%$ fewer tokens than the next-most compact baseline and over $98\%$ fewer tokens than sparse or hierarchical tokenizers, effectively reducing training memory by $40.4\%$ and inference time by $58.5\%$. Qualitative results further show improved preservation of thin structures, repeated components, and long-range connectivity.