Robot code tracks state to improve decisions and learn new tasks
Embodied Turing Machines: Stateful Code for Robot Recursive Self-Improvement
RoboticsComputer Vision and Pattern Recognition
Summary
Making robots that can improve themselves is hard because they often rely on AI models that guess what to do next. The authors suggest treating the robot and its surroundings like a giant computer tape, where all the important information is stored in clear, editable code. This code can use the robot’s camera and sensors to understand exactly what’s happening and make smarter decisions without relying on big AI models during tasks. Their approach lets robots handle many tasks, fix problems, and get better over time by updating the shared code they run.
What this means in practice
- •For robotics software engineers: Build robot controllers that use explicit code to track robot and environment state for faster, reliable decisions without AI models during operation.
- •For industrial automation teams: Develop reusable libraries to expand robot capabilities across multiple tasks with explicit state tracking and flexible recovery in manufacturing setups.
Authors
Kairui Hu, Siyuan Hu, Fangzhou Hong, Zhaoxi Chen, Ziwei Liu
Abstract
Most robot policies keep a model in the control loop: a VLA maps observations to actions, and an Agent Harness, such as Agent-as-Policy or Harness VLA queries a VLM for decision making at run time. We propose a different view: the embodied world is an Embodied Turing Machine, whose tape is the robot and environment state and rules are the policy. If this state can be represented accurately, the decision making can be written entirely in code. We therefore propose Code-Only-as-Policy (COAP): code measures and tracks the robot, environment, and task state from camera images and proprioception, and makes every decision from it. The same code applies across episodes, and different tasks share one library without a VLM or VLA in the loop. Compared with VLAs and Agent Harnesses, we analyze three advantages of COAP: (i) Explicit State: the state can be stored in code; (ii) Execution: code makes decision making controllable, recovers from failures flexibly, and runs fast and cheaply online; (iii) Extensibility: new tasks reuse, inherit, or extend the shared library, so capabilities can accumulate over tasks. These advantages make COAP a suitable medium for recursive self-improvement (RSI): coding agents develop the library in a closed loop, and each change is explicit and controllable. On RoboDojo's 42 bimanual tasks, the resulting library reaches a success rate of 70.24% without a model at test time. The upper bound of COAP lies in how accurately the state is represented for decision making and how robust the code logic is. We thus propose COAP as a new paradigm for embodied tasks; since it applies across episodes, it can also serve as an efficient data engine for VLAs and Agent Harnesses.