Text to 3d shape retrieval improved by geometry aware descriptors
Can One Hear Geometry? Gauge-Invariant Spectral Descriptors for Text-to-B-rep Retrieval
Computer Vision and Pattern Recognition
Summary
Finding 3D models from text descriptions is hard because shapes can be shown from different angles, sizes, or drawing styles. The authors built a new method that uses math to recognize shapes in ways that don't change when the shape is moved, rotated, or scaled. They create special shape summaries using geometry and structure, which helps match the right 3D solids to text much better than before. Their approach led to a big improvement in accuracy compared to prior methods.
What this means in practice
- •For 3d printing designers: Retrieve precise 3D solid models accurately from text queries despite different model orientations or scales.
- •For computer-aided design engineers: Search for CAD parts by their natural language descriptions with robust handling of shape transformations and modeling conventions.
Authors
Hongzhu Yi, Hao Liu, Tao Yu, Tiankun Yang, Yujia Yang, Zhenghao Zhang, Tianyu Zong, Zhenyu Guan, Bingkang Shi, Haopeng Jin, Yuanxiang Wang, Zhongtian Luo, Jungang Xu
Abstract
Text-to-3D solid retrieval requires aligning natural language with precise geometry despite variations in pose, scale, and recording conventions. Existing methods discretize Boundary Representation (B-rep) models into point clouds or meshes, or learn features from faces, edges, and topology, risking information loss or lacking explicit invariance guarantees. We propose a text-to-B-rep retrieval framework based on group actions and spectral geometric invariants, making invariance an explicit constraint on descriptor construction. Specifically, we use group actions to characterize equivalence transformations induced by translation, rotation, reflection, uniform scaling, and contour recording conventions. For sketch-extrusion solids, we construct a fixed 280-dimensional spectral geometric descriptor by integrating geometric information from three complementary levels, including cross-sectional regions, contour boundaries, and extrusion structures. Furthermore, we use descriptor-consistent multi-positive sets and geometric attribute supervision to promote structural alignment between text and solids. On Text-to-3D solid retrieval, our method achieves 88.6% Recall@5, exceeding the previous state of the art by 13.6 percentage points and demonstrating the effectiveness of explicit geometric invariants for precise 3D solid retrieval.