Agentic generation creates 400K articulated assets for simulation
RFW-A: Agentic Procedural Generation of 400K Articulated Assets over 500 Categories
Computer Vision and Pattern Recognition
Summary
Creating detailed 3D objects that can move and be used in simulations usually takes a lot of human effort. The authors found a way to teach AI agents to build general instructions (programs) for making whole categories of these objects. Once made, these instructions can quickly create many different objects without extra AI calls, saving time and cost. Using this method, they generated over 400,000 simulated articulated objects in 500 categories, which can help improve robots' understanding and interaction with real-world objects.
What this means in practice
- •For robotics engineers: Generate diverse simulated articulated objects efficiently for training robotic perception and manipulation systems.
- •For game developers: Create large libraries of sim-ready articulated models to populate interactive virtual environments with minimal manual modeling.
- •For simulation platform providers: Offer a cost-effective method to expand asset libraries, enabling more realistic physics-based interactions and policies transferable to real robots.$Commercial implications: Supports building and selling advanced simulation content and training environments for robotics customers.
Authors
Yuanbo Lyu, Changjian Pan, Saining Zhang, Yi Yang, Peishuo Li, Zhiheng Li, Jialu Liu, Junhao Chen, Jiawei Weng, Yupeng Gao, Weiqing Xiao, Wenyi Li, Jiarui Guo, Rui Zhang, Jingrun Du, Weiyi Lin, Zenan Meng, Hu Wei, Lin Qu, Zongzheng Zhang, Hao Zhao, Chaojian Li
Abstract
High-quality articulated assets are essential for embodied perception, simulation, and manipulation, yet constructing them at scale remains difficult. Manual annotation and procedural modeling require substantial human effort. Recent agentic systems automate asset construction through program synthesis, but repeat costly reasoning and refinement for every new asset. Our key observation is that assets within a category can share construction logic despite variations in structure and appearance. Agentic synthesis and procedural generation are therefore complementary: agents can construct reusable programs, while procedural execution reuses the resulting logic across many samples. We therefore present RFW-A, which combines the two in a new articulated-asset generation paradigm. Starting from existing articulated programs, RFW-A organizes reusable components and their dependencies into category-level generators. Agents test and refine these generators across sampled configurations and motion states. Once constructed and validated, the generators produce diverse assets through procedural execution with no further model calls or token charges. The estimated API cost of generator authoring averages $10.62, amortizing to $0.05 per asset over 200 assets, versus $1.24 for the sampled per-asset agentic synthesis baseline. Using RFW-A, we built RFW-A-400K, containing over 400K sim-ready articulated assets across more than 500 categories. Experiments show high structural and kinematic validity and stable execution across physics simulators. Beyond asset validity, training with RFW-A-400K improves part-level articulation estimation on seen and unseen categories, while policies trained in simulation with retrieved and program-edited assets transfer zero-shot to real articulated objects.