Behavior Prompting Policy: Demonstrations as Prompts for Manipulation
2026-06-29 • Robotics
Robotics
AI summaryⓘ
The authors explore 'behavior prompting,' a method that lets robots learn new tasks by watching just one human example during use. They created a new system called Behavior Prompting Policy (BPP) that turns a single demonstration and what the robot currently sees into actions. They also built a tool named iPhUMI to gather diverse training data, which helps the robot understand different tasks better. Finally, they tested their method on new drawing and manipulation tasks, showing it helps robots quickly adapt without costly retraining.
behavior promptingrobot learningvisuomotor policyin-context learningtask adaptationsingle demonstrationrobot manipulationtraining data diversitytest-time adaptation
Authors
Austin Patel, Ben Pekarek, Joel Enrique Castro Hernandez, Shuran Song
Abstract
We study behavior prompting, a paradigm that enables robots to perform new tasks at inference time given a single human demonstration, which we call a behavior prompt. To enable this capability, we present contributions in algorithm, data, and evaluation. For algorithm, we introduce Behavior Prompting Policy (BPP), an in-context visuomotor architecture that translates the behavior prompt and the current observation into robot actions. For data, we identify that task diversity is the primary driver of the prompting capability and introduce iPhUMI, a handheld manipulation interface for collecting diverse training data. For evaluation, we introduce DrawAnything and LIBERO-Gen to evaluate test-time adaptation to unseen drawing and tabletop manipulation tasks. We also demonstrate that iPhUMI serves as a practical interface for specifying behavior prompts at test time, enabling a human to command a robot via a single demonstration to complete known tasks or to define new robot capabilities. Altogether, behavior prompting provides a flexible and scalable way to teach robots new skills without the need for expensive fine-tuning. Our project website is located at https://behavior-prompting.github.io/ .