Agent performance drops in long and complex robot tasks
LIBERO-Agent: Evaluating General-Purpose Agents for Direct Embodied Manipulation
Robotics
Summary
Robots controlled by general-purpose AI agents can do well on easy tasks and simple perception, but struggle with harder and longer tasks that involve more steps. The authors created a new benchmark called LIBERO-Agent to test how well these AI agents handle robot manipulation in many tasks. They found that richer information helps with short tasks, and some agents do better depending on how they learn from demonstrations. Among the tested agents, GPT-6 Astra performed best, especially when the task required steady contact with objects, but it still made mistakes due to confusion between steps and spatial errors.
What this means in practice
- •For robotics engineers: Evaluate general-purpose AI agents for robot control and identify performance limits in complex manipulation tasks.
- •For automation system designers: Design systems that integrate AI agents with varying observation richness to improve robot task success rates.
Authors
Zijie Diao, Yitong Chen, Sicheng Xie, Tianyi Lu, Wujian Peng, Guojin Zhong, Houze Xu, Ziyi Ye, Zuxuan Wu, Yu-Gang Jiang
Abstract
General-purpose agents can plan, use tools, and revise their behavior from feedback, but it remains unclear whether these capabilities transfer from digital environments to embodied manipulation. To investigate this question, we introduce LIBERO-Agent, an agent-native benchmark for evaluating these agents in robot manipulation tasks. Rather than asking agents to submit task-level Python control programs or operate through high-level robot skills, LIBERO-Agent provides an interactive robotic environment where agents can select which observations to inspect, process them with their own tools, and issue native action commands. LIBERO-Agent integrates 200 tasks into a common interaction framework and provides a 30-task primary suite that separates perception, short-horizon execution, and long-horizon composition. Results reveal a pronounced reliability gap: while agents perform well on perception and easy short-horizon tasks, their performance degrades substantially on hard short-horizon and long-horizon tasks. Richer observations improve short-horizon manipulation, while demonstration benefits depend on the agent and format. Among these agents, GPT-6 Astra achieves the strongest overall performance. Further analysis shows its major advantage lies in mechanism interaction, especially when sustained physical contact is needed, while its remaining failures stem from cross-stage interference and geometric errors.