Physical coding agents improve robot task success with evolving programs
Self-Evolving Coding Agents: From Digital Programs to Physical-World Intelligence
Robotics
Summary
Robots often struggle with small changes in their environment or instructions because their actions are hard to adjust and understand. To fix this, the authors suggest representing robot tasks as computer code that tracks objects and steps clearly. They built a system called HexaAnything that uses this idea to plan, check, and fix its actions, learning from experience like rewriting its own code. This method helped robots do better on new tasks and complete complex experiments faster than before. The approach also lets the system improve itself by evolving its knowledge, models, and hardware designs over time.
What this means in practice
- •For robotics engineers: Use physical coding agents to improve robotic manipulation and task success under changing environments and instructions.
- •For manufacturing automation teams: Deploy self-evolving robotic agents that adapt workflows and hardware designs to optimize production and reduce manual reprogramming.$Commercial implications: Enables adaptive manufacturing robots that evolve task handling capabilities and hardware setups, enhancing productivity and reducing downtime.
Authors
Hongcheng Gao, Jingjing Zhou, Zelin Zheng, Shijia Ge, Jay Zhu, Yazhe Wang, Jianshu Zeng, Xuan Shangguan, Di Wu, Lingyu He, Zhiqi Jia, Sihang Wu, Xiao He
Abstract
Vision-language-action (VLA) and world-action (WAM) models map observations and instructions directly to robot actions. This directness ties a policy to training: minor layout or viewpoint changes cause failure, and instructions generalize poorly. The root cause lies in representation: task requirements, conditions, progress, and failure recovery are implicitly encoded in action sequences, making them difficult to inspect or revise. Digital coding agents offer a precedent: LLMs call tools, verify results, and revise from feedback as executable code. The same working pattern of explicit state, manageable execution, and revisable procedures underlies generalization and long-horizon execution in the physical world, letting physical experience return as reusable programs, memory, or evidence. We propose Physical Coding, representing task state and execution as code. Code as World records objects, relations, constraints, and progress; Code as Policy organizes planning, verification, recovery, and execution. We build HexaAnything, which calls perception, planning, and control tools, including VLA/WAM policies, and makes in-the-loop decisions from external feedback. Verified traces become data and memory, enabling evolution from tools and Harness to model weights, architectures, and ultimately hardware and task design. On RoboCasa365, HexaAnything improves Composite-Unseen and overall success over XR-1 VLA, and its Harness-trained HexaModel beats the base on every split, indicating code traces internalize physical execution. On PhyBench and a dual-arm AgileX robot, the agent autonomously completes physics experiments and most tabletop tasks, often faster than published results. We observe data, model, and tool self-evolution; future work targets weight internalization, autonomous redesign of architectures, languages, representations, and tasks, and deployment in manufacturing and science.