Summary
Automating professional engineering work is hard because it needs careful thinking about shapes and physical rules that stay consistent across many steps and tools. The authors created EngiWorld, a big test with over 1,300 tasks across six engineering areas and many software programs to see how well current AI agents can handle these jobs. They built a system that checks whether the designs made by AI follow the rules, are physically possible, and meet specifications. Testing top AI models showed they still struggle a lot, with few tasks done successfully. EngiWorld is the first solid way to measure how close AI is to doing real engineering work end to end.
What this means in practice
- •For industrial engineers: Use EngiWorld to benchmark and improve AI tools that assist with complex design tasks across multiple engineering software.
- •For software developers for engineering tools: Develop and test AI integration features in engineering software by evaluating them against EngiWorld’s comprehensive task suite and artifact checks.
Authors
Hongcheng Gao, Hailong Qu, Yu Lei, Henghui Sun, Haoyang Li, Yipeng Wei, Naihao Xue, Xiaohan Yu, Zhuo Tao, Yihe Zang, Yajiao Wang, Jingyi Tang, Yi Li, Jingjing Zhou, Jie Luo, Bohan Zeng, Chengyu Shen, Hao Jiang, Chong Chen, Bowen Qu, Olive Huang, Zeqiang Wang
Abstract
Autonomous agents have made rapid progress in general-purpose computer use, but reliable automation of professional industrial engineering remains out of reach, as engineering workflows demand reasoning over geometric and physical constraints and dependencies preserved across software and design stages. We present EngiWorld, the first benchmark structured around the complete design loop: 1,301 expert-curated tasks spanning 6 engineering domains (CAD, CAE, CAM, BIM, EDA, and 3D visualization) and 26 professional software platforms, with both GUI and CLI interfaces and 6 task types ranging from software-selection to open-ended tasks. We further introduce an artifact-centric evaluation methodology built on a unified domain-verifier suite, which programmatically checks the geometric validity, physical feasibility, and rule compliance of final and intermediate artifacts, and scores quantitative design tasks continuously by specification attainment rather than binary success. Evaluation of seven frontier models reveals a substantial capability gap: the strongest model achieves an EngiScore of only 44.3, and just 3.6% of multi-software attempts succeed. EngiWorld provides the first rigorous foundation for measuring progress toward agents that operate professional engineering software end to end.