GPT-6 robot agents succeed more using fewer tokens and calls
Fewer Tokens, Better Action: GPT-6 Astra Robot Agents with 14% Higher Success Rate but 65% Fewer Tokens
Computer Vision and Pattern Recognition
Summary
Robots controlled by vision-language models often waste many words and repeated information when figuring out what to do next. The authors created a new system called PyRUA-Lean that carefully chooses when to ask for new observations and runs small computer programs to check and retry actions locally. This method helps robots complete more tasks successfully while using much fewer words and fewer times to ask questions. Their tests on hundreds of robot tasks showed a clear improvement in success rate and efficiency compared to a standard approach.
What this means in practice
- •For robotics engineers: Improve robot task success rates and reduce computational overhead by selectively requesting observations and combining classical and learned control actions.
- •For automation system developers: Build more efficient interactive robot controllers that require fewer language model invocations, lowering latency and resource use in complex automation setups.
Authors
Ruiyang Si, Jianxin Bi, Shunyu Yang, Rui Ni, Wenbo Huang, Qiang Wang, Shulong Jiang, Duomin Wang, Xiuyu Li, Haiwen Feng, Zhen Dong, Daquan Zhou
Abstract
Vision language model (VLM) agents can control robots through visual feedback and action primitives, but repeated model invocations and redundant observations incur substantial token overhead. We introduce PyRUA-Lean, an interactive code-execution framework that couples feedback-driven primitive composition with selective observation: the agent composes classical robot primitives and learned vision-language-action (VLA) policies into Python cells that perform conditional checks and local retries, returning only explicitly requested images and state feedback for replanning. Across 700 simulated task instances from LIBERO-PRO, RoboTwin 2.0, and RoboCasa365, we compare PyRUA-Lean with a tool-calling baseline using the same GPT-6 Astra planner and underlying robot primitives. Under equal LLM-call budgets, PyRUA-Lean increases overall success from 63.1% to 71.7%. On instances solved by both agents, it uses 49% fewer LLM calls and 65% fewer input tokens.