Uranus builds fast scalable robot simulation with online control
Uranus: Building the Next-Generation Simulation Infrastructure for Embodied AI
RoboticsArtificial Intelligence
Summary
Simulating how robots move and interact with their environment takes a lot of work and time. The authors created Uranus, a new kind of robot simulator that uses a special machine learning model to generate robot movements smoothly and quickly as it receives commands in real time. Uranus can produce realistic video frames at 24 frames per second and supports many different robot types and camera views. The authors tested the system thoroughly and shared the code and model so others can use and improve it.
What this means in practice
- •For robotics engineers: Generate large-scale realistic robot motion data for training control algorithms without costly real-world trials.
- •For robot software developers: Simultaneously create synchronized multi-view video outputs from diverse robot types to test and debug robot vision and control software.
Authors
Wenkang Qin, Yukun Zhou, Noah Shen, Jisong Cai, Dongxiao Mao, Baicheng Li, Yue Zhang, Wei Sui
Abstract
Scalable simulation is essential for robot data generation, policy training, evaluation, and safe iteration, yet real-world interaction is costly and conventional simulators require labor-intensive construction. We present Uranus, a data-driven robot simulator built around a joint-trajectory-conditioned autoregressive diffusion model. Uranus offers three key capabilities: (1) streaming, open-ended rollout, which receives future joint-position trajectories online and autoregressively generates one latent frame per step, corresponding to four RGB frames, without a fixed horizon; (2) low-latency generation, achieving 24 FPS after inference optimization; and (3) scalable, extensible robot control, providing a unified interface for synchronized multi-view generation across diverse robot embodiments and camera configurations. We conduct comprehensive quantitative and qualitative evaluations on both in-distribution and out-of-distribution data, providing an objective assessment of Uranus and clearly identifying its current limitations. We release the code and model weights to empower the community with practical tools and insights.