Robot manipulation improves by using belief based imitation learning
BEACON: Belief-Enabled Adaptive CONtrol for Imitation Learning under Uncertainty
Robotics
Summary
Robots sometimes struggle to do tasks because they can’t directly see all the important details about what’s happening. The authors tackle this by teaching robots to imagine what the hidden details might be, instead of relying just on recent observations. Their method helps the robot decide when to try new actions to learn more and when to stick with what it thinks is best. Tests showed this new approach works much better than using observations alone and almost as well as if the robot had perfect information.
What this means in practice
- •For robotics engineers: Develop control policies for robot manipulators that improve handling of uncertain or hidden object states in physical tasks.
- •For automation system designers: Design automated systems able to better infer hidden states during sequential interactions, improving reliability in manufacturing or service tasks.
Authors
Moonyoung Lee, Soumojit Bhattacharya, George Kantor, Oliver Kroemer
Abstract
Robot manipulation tasks often involve hidden state information that cannot be directly observed and must be inferred through sequential physical interactions. In such partially observable settings, conditioning an imitation learning policy directly on the recent raw observation history leads to poor performance. This is due to state aliasing, wherein identical observations may arise from different hidden states, and the policy receives conflicting action labels for the same input. To enable history-aware disambiguation capability, we propose conditioning a diffusion policy on a structured representation of the hidden states using Bayesian belief that exposes both the current most likely state estimate and the remaining uncertainty. This representation replaces raw history with a structured, compact input, enabling the policy to implicitly modulate between exploratory and exploitative behaviors based on belief uncertainty, without explicit mode switching or reward shaping. We evaluate across two domains with qualitatively different belief representations: a continuous belief for cornstalk gripper alignment via tactile sensing, and a discrete categorical distribution for latched door opening. In both domains, the belief-conditioned policy substantially outperforms the observation-only baseline and approaches privileged ground-truth performance, with ablations illustrating that the policy adapts its exploration behavior depending on the belief uncertainty at inference time.