Robot control adapts by knowing both user intent and robot ability
Capability-Aware Arbitration for Semantic Intent-Based Shared Control
Robotics
Summary
Robots that help people by guessing what they want can sometimes try to do too much and make mistakes if they are not good at the task. The paper shows a way for the robot to consider not just what the human wants but also how well the robot can do the job at that moment. This helps the robot decide when to take control and when to let the human do it, making tasks like picking and stacking objects easier and more successful. Tests with people showed that this method works better than older ways of sharing control between human and robot.
What this means in practice
- •For robotics engineers: Build robots that smartly balance user commands and robot skill for better pick-and-place tasks.
- •For manufacturing automation teams: Develop shared-control systems that improve stacking and manipulation by adjusting robot authority based on task difficulty.
Authors
Zhaoda Du, Michael Bowman, Xiaoli Zhang
Abstract
Shared control often allocates robot authority based on confidence in inferred human intent, assuming reliable autonomous execution. When this assumption fails, high intent confidence can cause over-helping. We present a capability-aware shared-control framework in which a vision-language model (VLM) infers human intent and provides semantic-intent confidence, while a vision-language-action (VLA) policy generates autonomous actions. VLA capability confidence is estimated online from the dispersion and local instability of stochastic action trajectories. We design a nonlinear arbitration policy that combines Bayesian-filtered semantic-intent confidence with VLA capability confidence through a sigmoid mapping to adapt robot authority. Our evaluation combined VLM/VLA confidence assessment with a study involving 12 participants performing pick-and-place and bidirectional stacking under in-distribution and out-of-distribution conditions. The proposed method achieved the highest task success rate (92%), compared with manual teleoperation (83%), intent-only arbitration (44%), and fixed equal-weight blending (10%). It also achieved higher control friendliness and lower authority-weighted disagreement than both shared-control baselines. These results demonstrate the benefit of incorporating VLA capability into authority allocation to mitigate over-helping and improve shared-control performance.