BT-Agent improves onboard robot task handling with behavior trees
BT-Agent: Deploying Onboard Robot Agents with Behavior Trees and Multi-Fidelity Evaluation
Robotics
Summary
Robots that work on their own need to think about tasks and act quickly, even with limited computing power. The authors created BT-Agent, a system that helps robots use a language model to make decisions while relying on predefined skills that guide their actions and catch problems. They tested this on a walking robot doing different tasks and showed it could stop or retry actions on its own. The authors also built a big test set to compare robot agents more easily and found BT-Agent works better than other methods, especially when using bigger language models.
What this means in practice
- •For robot developers: Deploy onboard agents that combine language models and behavior trees for improved task execution and safety checks on mobile robots.
- •For simulation engineers: Use a multi-fidelity benchmark to efficiently test and compare robot agent behaviors across text and simulated environments.
Authors
Fabio Hübel, Adam Lilja, Yue Yin, Siming He, Claire Tomlin, Shankar Sastry, Jonas Frey, Marco Pavone
Abstract
Deploying language-model agents onboard mobile robots requires task reasoning and responsive execution within limited compute and memory. We propose BT-Agent, an onboard architecture combining LLM-driven task decisions with predefined behavior-tree (BT) skills. These skills enforce execution conditions and return structured failure feedback. We demonstrate fully onboard operation with a 9B model on a quadruped across five task categories. Intervention tests show that local checks stop ongoing navigation without waiting for another model decision, while the agent remains active to retry or report constraints. To compare agent designs efficiently, we introduce a 161-task multi-fidelity benchmark, evaluated in text on all tasks and in Habitat on 39. Text supports 10.9$\times$ higher evaluation throughput, while simulation reveals how perception and motion failures change task outcomes. BT-Agent's largest success gains over the strongest Act/ReAct baseline occur at 4B and 9B: approximately 10 and 5 percentage points. Ablations identify structured tool calls and informative failure feedback as the most useful components.