Robot environment edits help learn clearer preferences from humans
Environment Edits as a Corrective Signal for Reward Learning
Robotics
Summary
Robots can be corrected by humans to improve how they behave, but one correction can change many things at once, which is confusing. The authors suggest letting humans also change the environment around the robot without moving the robot itself, making it clear what object or feature the human cares about. They created a system called FERRET that learns from both ways of correcting the robot, improving how well robots understand what humans want. This method helps robots learn preferences that were unclear from just watching robot movements.
What this means in practice
- •For robotics engineers: Use environment edits alongside robot movements to improve robot learning of human preferences in object handling tasks.
- •For interactive system designers: Incorporate environmental scene changes as a form of user feedback to clarify ambiguous corrections in robot behavior adjustment systems.
Authors
Isaac Ngui, Courtney McBeth, Nancy M. Amato
Abstract
Robot corrections allow people to adjust a robot's behavior, but a single correction changes many features. Also, preferences about object attributes and relationships between objects cannot be expressed by moving the robot. We propose environment edits as a complementary form of feedback. The robot's motion is held constant as the human edits the scene, so the only explanation for the edit is the moved object. We introduce Feature Learning from Environmental and Robot-Relational Edit Traces (FERRET), which learns preferences from both robot and environment corrections. We show that environment edits are able to disambiguate features and learn preferences that robot corrections alone cannot.