Particle filter helps assist with changing goals during tasks
Assisting for Open-Ended Tasks: Goal-Oriented Shared Autonomy as a Particle Filter
Robotics
Summary
Shared autonomy systems help people finish tasks by guessing what they want to do and assisting them. But older systems need to know all possible goals ahead of time, which is limiting. The authors made a new method that can guess new goals as the task changes by using a particle filter to keep track of many possible goals. Their system uses AI models to understand the situation and offers helpful actions, adjusting based on human input. Tests showed this approach made tasks faster and people felt happier using it.
What this means in practice
- •For robotics engineers: Build assistive robots that can adapt to users' changing goals without needing a fixed set of targets.
- •For industrial automation teams: Deploy teleoperated systems that reduce operator time and improve satisfaction by dynamically understanding task goals.
- •For smart home device developers: Create adaptive home assistant robots that interpret open-ended user intents during daily activities.$Commercial implications: Enables commercial home assistant robots to offer dynamic context-aware help, improving user experience and allowing new market products.
Authors
Mengxue Fu, Ethan Xu, Sam Iyer-Singh, Yinlong Dai, Michael Hagenow, Dylan P. Losey
Abstract
A common approach for shared autonomy blends human inputs with autonomous assistance based on the human's likely goal. However, most existing approaches assume that a static set of possible goals is known a priori, which limits the use of such methods in unstructured assistive settings. We instead investigate how to enable shared autonomy with open-ended and dynamically changing goals. We formulate goal-oriented shared autonomy as a particle filter in which particles represent candidate human goals. Unlike conventional approaches with a fixed goal set, our transition model dynamically proposes new candidate goals as the interaction evolves, and human actions update the belief over these goals in real time. We instantiate this framework with foundation models (e.g., vision grounding and large language models) that propose context-relevant semantic goals, generate goal-conditioned assistance from low-level skill primitives, and refine those skills from human corrections. We assess our approach through a user study where 12 participants perform a variety of tabletop manipulation tasks with our method and state-of-the-art shared autonomy baselines. The results show that our particle filter-based approach reduces the amount of time users spend teleoperating the system and improves user satisfaction. User study videos: https://youtu.be/Ii26XuRqm9c