Human-robot interaction improves by learning user preferences faster
Simultaneous Forward and Inverse Human-in-the-Loop Optimization
Robotics
Summary
Getting robots to work well with humans requires understanding what people want, but this is often tricky because preferences differ and feedback is costly to get. The authors introduce a method called SFIHILO that learns what a user values by simultaneously predicting outcomes from robot actions and asking smart questions to the user to quickly improve its understanding. This approach reduces the need for many trials and helps find the best robot behavior for each person more efficiently. In simulated tests, it worked well despite noise and user differences, making personalized robot assistance more practical.
What this means in practice
- •For robotics developers: Design assistive robots that rapidly learn individual user preferences to improve personalized support without extensive trial-and-error.
- •For wearable device engineers: Build adaptive devices that infer wearer comfort or performance goals quickly through minimal user feedback and optimize control accordingly.
Authors
Kyeongwon Park, Steven H. Collins
Abstract
Subjective user experience is important to human-robot interaction, but the outcomes users value, and how those preferences vary across individuals and contexts, are often unknown. While inverse learning approaches using human data can help identify user rewards, in many assistive settings the experimental costs of executing a control policy, measuring biomechanical or physiological outcomes, and collecting user feedback often limit the number of queries and optimization iterations. Here, we present Simultaneous Forward and Inverse Human-In-the-Loop Optimization (SFIHILO), which efficiently infers individual-specific reward functions from preferences over human outcomes and identifies a final control policy that maximizes the learned reward. SFIHILO bootstraps a forward model to predict user outcomes from control policies, uses this model for active querying to accelerate inverse reward learning, and then optimizes the final control policy without additional user trials. We score candidate policies by their expected reduction in uncertainty across both forward and inverse beliefs, targeting a regional preference boundary to robustly inform this simultaneous learning process. In simulation, we show that SFIHILO was effective across user heterogeneity, outcome dimensionalities, precision requirements, noise levels, and nonstationarity; compared with mutual information approaches, the proposed active querying strategy significantly improved sample efficiency in inverse learning while preserving forward model accuracy. This approach demonstrates the potential to infer latent human goals, enabling more transferable and effective human-robot interaction.