CarryOn enables personalized and safe robot assistance in real environments
CarryOn: A Dual-Loop Harness for Personalized and Reliable Robot Assistance
RoboticsComputer Vision and Pattern Recognition
Summary
Robots today can follow general instructions but often lack the ability to adapt to individual user preferences or guarantee safe and reliable actions. The authors developed CarryOn, a system that helps robots remember user-specific ways to complete tasks and watch for potential hazards during execution. If something goes wrong, CarryOn can recover from errors using learned safety techniques. This approach was tested on real and simulated robots, showing fewer mistakes and better recovery, making robot helpers more personal and dependable.
What this means in practice
- •For robotics engineers: Improve robots’ ability to personalize task execution and avoid hazards when interacting with users in dynamic environments.
- •For manufacturing operators: Use safety-supervised robot assistants that adapt to operator preferences and recover from physical failures during production tasks.
Authors
Zixuan Wang, Yuqi Liu, Yuxin Chen, Sengqiao Yang, Jinhui Ye, Pengguang Chen, Changsheng Lu, Shu Liu, Bei Yu, Jiaya Jia
Abstract
While generalist vision-language-action (VLA) models have enabled diverse open-world manipulation, real-world robotic assistance requires two critical capabilities they currently lack: personalization and reliable execution. First, executing an instruction like "make me a sandwich" requires grounding subtask decomposition in persistent user preferences rather than relying on a single generic solution. Second, reliable deployment demands explainable safety assurance that proactively prevents hazards and gracefully recovers from occasional physical failures. To address these gaps, we present CarryOn, a dual-loop harness designed to augment a frozen VLA policy. The planning loop maintains persistent user-level and scene-level context that is continually grounded against new observations and revised through interaction, enabling personalized task selection and execution grounding across conversations and changing scenes. The execution loop actively supervises policy rollouts, shielding actions from predicted hazardous contact. When execution fails, the system invokes reusable control-based recovery skills that restore task progress and are refined through reflection on earlier failures. On a real robot and in simulation, CarryOn dispatches preference-consistent subtasks across users and scenes, reduces the violation rate of an already safety-finetuned VLA from 52.6% to 6.1%, and increases recovery success from 13% to 80% when refined skills are reused.