Llms guided by geometry improve text to cad model generation
CAD-WAM: Anticipating Geometry for Text-to-CAD Generation
Computer Vision and Pattern RecognitionGraphics
Summary
Making editable 3D CAD models from text is hard because one wrong step can mess up the whole design. The authors developed CAD-WAM, which uses a language model to choose the next modeling action based not only on words but also on the current shape of the model. Their system checks its work by predicting what the model should look like after each step, improving accuracy and stability. This method created better and more reliable CAD models from text descriptions than previous approaches.
What this means in practice
- •For industrial designers: Generate accurate editable CAD models from text descriptions for rapid design iteration.
- •For architectural drafting teams: Convert textual project requirements into reliable intermediate CAD designs to speed early-stage modeling.
Authors
Zhengyang Li, Weijie Zheng, Yuxin Cui, Sijie Wang, Wenjie Feng, Qiyu Kang
Abstract
Generating editable computer-aided design (CAD) models from text requires selecting operations that produce the intended changes to the geometry constructed so far. An incorrect operation can compromise subsequent steps, making reliable multi-step generation challenging. Existing methods based on large language models (LLMs) primarily learn to predict commands, without explicit supervision of the geometric states those commands should produce. Inspired by world-action models, we introduce CAD-WAM, which couples geometry-conditioned action generation with next-state prediction. We define complete modeling operations as actions in a CAD domain-specific language (DSL) and encode surface point clouds of intermediate solids into compact latent states using a pretrained variational autoencoder. An LLM backbone generates the next action from the target description and current geometric state, while an auxiliary head on the same backbone predicts the next state to provide geometric supervision during training. We construct step-wise training trajectories from parametric CAD models in the DeepCAD dataset, pairing each trajectory with an expert-level textual description. At inference, each generated action is executed by a CAD kernel, and the resulting solid is re-encoded to inform the next decision. On the DeepCAD test set, CAD-WAM outperforms the task-specific baselines in geometric accuracy and output validity, with larger gains on models requiring more operations, and produces more accurate geometry than general-purpose LLMs prompted with the same DSL. Ablations further show that next-state supervision reduces mean Chamfer distance by 13.2\% relative to an otherwise identical model trained only to predict actions. Code is available at https://github.com/Lzy-1037/CAD-WAM.