Humanoids benchmark tool use from choice to motion execution
HumanoidToolBench: Benchmarking Humanoid Tool Use from Selection to Mobile Execution
RoboticsArtificial Intelligence
Summary
Robots shaped like humans need to use tools to do tasks they can't do with just their bodies. To do this well, they must pick the right tool and then move and use it properly. The authors created a set of 18 different tasks to test how well humanoid robots can pick and use tools while moving. They also collected over 3,000 examples of these tasks being done on both simulations and a real robot. Their tests show robots still struggle with picking the right tools and finishing the tasks correctly.
What this means in practice
- •For robotics development teams: Evaluate and improve humanoid robot abilities in tool selection, manipulation, and movement using a comprehensive multi-task benchmark with real and simulated data.
- •For industrial automation teams: Test mobile humanoid robots on complex tasks requiring tool use to enhance autonomous operation in real-world environments.
Authors
Kyochul Jang, Seohyeon Park, Ohchul Kwon, Sangjun Park, Junhyeok Choi, Seungyeop Yi, Chaeyun Kim, Sangkyu Lee, Idan Szpektor, Avi Caciularu, Jongmin Park, Youngjae Yu
Abstract
As robotic hardware and learning methods advance, humanoids need tools to perform tasks beyond their inherent physical limits. Successful tool use requires selecting a suitable tool and coordinating manipulation and, when needed, locomotion to complete the task. Existing benchmarks do not jointly evaluate these capabilities on a humanoid. We introduce HumanoidToolBench, an 18-task benchmark spanning three scenarios, three execution levels, and two tool-set modes, together with ToolBook, a dataset of 3.1k demonstrations collected in simulation and on a real Unitree G1. Evaluation of seven policies in simulation and three on the real robot reveals substantial gaps between selecting a suitable tool and completing the task. Focused GR00T N1.7 probes show reduced selection accuracy on unseen tools and continued task execution under unrelated instructions. Code and data are available at https://snu-pi.github.io/HumanoidToolBench/.