Geometry aware method improves 3d cad model reconstruction from images
VGGT-CAD: Reconstructing Parametric CAD 3D Model with Geometric Grounding
Computer Vision and Pattern Recognition
Summary
Turning images or videos into editable 3D CAD models is hard because it requires understanding both detailed shapes and how those shapes were built. The authors created a new method called VGGT-CAD that uses geometric clues from multiple camera views to better recover 3D models and the steps to make them. Their system can choose the best camera angles to look at and combines features from these views to create accurate CAD instructions. They also built a new dataset with videos of CAD models to test and improve their method. Experiments show VGGT-CAD works well from single or multiple views.
What this means in practice
- •For product designers: Automatically generate editable 3D CAD models from product images or videos to speed up design iteration.
- •For architectural visualization teams: Reconstruct precise parametric building components from multi-view photos for parametric modeling and modification.
Authors
Chunan Yu, Tianrun Chen, Fu Shen, Cheng Chen, Lanyun Zhu, Yang Yang
Abstract
Parametric CAD reconstruction requires recovering both precise geometry and editable modeling operations from visual observations, making it challenging under limited and ambiguous views. Existing methods mainly rely on 2D appearance cues and lack strong multi-view geometric priors. In this work, we present VGGT-CAD, a geometry-aware framework for parametric CAD reconstruction from single- and multi-view observations. We transfer pretrained 3D geometric priors into CAD reconstruction by encoding camera parameters as condition tokens and jointly modeling them with image tokens. To handle varying numbers of viewpoints, we introduce a variable-view cross-view context aggregation module that adaptively fuses multi-view features. We further develop a training-free geometry-aware view selection strategy to select complementary and reliable frames during inference. The resulting representation is decoded into CAD command sequences using a non-autoregressive decoder. We also develop VideoCAD, a large-scale multi-view video benchmark derived from existing CAD data through multi-view re-rendering. Extensive experiments demonstrate the effectiveness of VGGT-CAD for visual CAD reconstruction under different observation configurations.