MatchingPolicy: Correspondence-Aware Policy Enables Cross-Object In-Context Learning
2026-08-17 • Robotics
Robotics
AI summaryⓘ
The authors address a problem where robots learn to imitate tasks but struggle when faced with new objects or situations. They propose MatchingPolicy, a method that separates the task of matching parts of a demonstration to a new scene from the task of deciding what actions to take. Their approach uses detailed visual connections to guide the robot's actions, improving its ability to handle unfamiliar objects. Tests in both simulated and real-world settings show that their method helps robots learn better from few examples and adapt to new challenges more reliably.
In-context imitation learningFew-shot learningPolicy generalizationSemantic correspondenceDiffusion policyRobotic manipulationVision foundation modelsMatching algorithmOut-of-distribution transferRLBench
Authors
Qijin She, Hanyang Yu, Zeming Li, Ping Tan
Abstract
In-context imitation learning enables few-shot policy generalization but struggles to maintain performance on unseen objects and novel scenarios. To address this, we introduce MatchingPolicy, a correspondence-driven framework that explicitly decouples demonstration-to-scene matching from policy learning. Central to our method is a correspondence-aware diffusion policy that conditions robotic actions directly on dense semantic correspondences. This architectural separation resolves the inherent conflict between correspondence identification and action adaptation, enabling robust out-of-distribution transfer. Our framework integrates vision foundation models with a novel two-stage matching algorithm to dynamically establish reliable correspondences. Extensive evaluations on RLBench and real-world manipulation tasks confirm that MatchingPolicy achieves superior few-shot performance, generalizing reliably across unseen object instances and semantic categories.