AI summaryⓘ
The authors introduce RynnValue, a new model that helps robots learn tasks by measuring how far they are in time from a goal, rather than relying on preferences or progress labels that don't work well across different robots and tasks. They used over 7,000 hours of data with time-based labels, allowing the model to learn effectively without human preference input. Their techniques help the model avoid mistakes like ignoring failures, leading to better predictions and performance. When applied, RynnValue improved robot success rates significantly in both real-world tests and offline evaluations. Overall, the authors show that using temporal distance as guidance is a scalable and practical way for teaching robots general skills.
robot learningreward modeltemporal distancerobotic manipulationvalue functionpreference supervisiontemporal samplingdense rewardspotential-based shapinggeneralist policies
Authors
Dongchi Huang, Hongyin Zhang, Bohan Hou, Siteng Huang, Zhian Su, Hang Guo, Tong Lu, Zhaofeng Xu, Jiahao Tang, Jianfei Yang, Donglin Wang, Peixi Peng, Mingxiu Chen, Deli Zhao, Xin Li
Abstract
General-purpose reward models are increasingly the bottleneck for scaling robot learning, yet the recipe for learning value-related capabilities from large-scale heterogeneous corpora remains underexplored. Existing approaches tie supervision to task-internal anchors such as preferences or normalized progress, none of which transfer cleanly across embodiments and data sources. We introduce RynnValue, an open-source value foundation model for robotic manipulation that replaces these anchors with temporal distance, the directed cost-to-go from an observation to the language-specified goal. Because temporal-distance labels can be derived directly from timestamps, RynnValue scales to over 7,000 hours and roughly 3M instruction-conditioned clips without preference or progress annotations. To make temporal-value learning reliable at scale, we combine random temporal sampling, temporal-order shuffling, and value-isolation attention, suppressing shortcuts that would leave predictions insensitive to failures and regressions. Trained without preference labels, RynnValue attains an average Kendall's tau_a of 0.675 on RBM-EVAL-OOD, surpassing the fully preference-supervised state of the art (0.655) and more than doubling a progress-only counterpart (0.292), while generalizing zero-shot to unseen tasks, embodiments, and viewpoints. Converted into dense rewards via potential-based shaping, it raises real-world policy success from 52.5% to 72.5% online and from 63.8% to 82.5% offline. These results establish temporal distance as a scalable supervision target and practical reward interface for generalist robot policies.