GIFT: Guided Intermediate Feature Training via Action-Oriented Structural Supervision for Robotic Manipulation
2026-09-03 • Robotics
Robotics
AI summaryⓘ
The authors address a problem where robot vision systems see too much detail that isn't useful for controlling actions, which they call the "action-sufficiency gap." They propose a method named GIFT to guide the robot's internal features to focus on important control-related aspects like object shape, what can be used or interacted with, and task goals. They test their approach on different robot control models and find that GIFT improves performance on several robotics tasks, especially those involving complex objects and precise actions. Their work shows that teaching robots to focus on functionally relevant features helps them perform better in new and challenging environments.
vision-language pre-trainingrobotic manipulationaction-sufficiency gapintermediate featuresgeometry alignmentaffordance predictiongoal-region reconstructionworld-action modelzero-shot transferarticulated-object tasks
Authors
Yupeng Zheng, Xiang Li, Songen Gu, Yuhang Zheng, Shuai Tian, Weize Li, Linbo Wang, Chaoyue Li, Qichao Zhang, Haoran Li, Zhongpu Xia, Ya-Qin Zhang, Shuicheng Yan, Dongbin Zhao
Abstract
Vision-language pre-training and predictive world modeling provide robot policies with rich semantic and dynamic visual features, but their native action and visual-prediction objectives may omit critical physical and task structure while retaining control-irrelevant visual redundancy. We call this mismatch between visual richness and control utility the action-sufficiency gap. We investigate whether this gap can be bridged by guiding intermediate features to preserve three control-relevant structure in robotic manipulation: geometry governing motion feasibility, affordance encoding instruction-relevant entities, and goals grounding instructions in task-relevant regions. To this end, we present GIFT (Guided Intermediate Feature Training), an architecture-flexible framework for learning intermediate features that translates these structures into training-time constraints through geometry alignment, affordance prediction, and goal-region reconstruction. We instantiate GIFT in a Vision-Language-Action (VLA) policy, a direct-action World-Action Model (WAM), and an inverse-dynamics WAM while retaining each model's action formulation. Under zero-shot transfer to LIBERO-Plus, GIFT-VLA, GIFT-WAM-Fast, and GIFT-WAM-IDM outperform StarVLA-OFT, Fast-WAM, and Fast-WAM-IDM by 4.6, 12.6, and 5.2 points, reaching 79.6%, 72.6%, and 87.8%, respectively. On RoboCasa, the three GIFT variants reach 61.4%, 83.6%, and 82.3%, outperforming their counterparts by 12.6, 9.0, and 8.4 points, respectively. Together, these results establish learning functionally structured intermediate features as a reusable principle across model-specific action formulations, with especially large gains on articulated-object tasks and high-precision real-world manipulation under unseen visual and spatial perturbations. Project page: https://openphoenix-team.github.io/GIFT-pages.