Generalist robot policies guide fast learning of specialist robots
FIERCE: From Generalist Robot Policies to Fast Specialists via Progress-Failure Feedback
Robotics
Summary
Training robots to do specific tasks using limited physical practice is tricky because they need detailed feedback during learning. The authors present FIERCE, a technique that uses a general robot policy as a starting point and a special learner that watches progress and failure signals to guide training. This method doesn't require ongoing help from the general policy or complex simulations and keeps only the small, specialized robot program at the end. The authors tested their approach in both computer simulations and real-world contact-heavy tasks, showing how it improves robot learning efficiency.
What this means in practice
- •For robotics engineers: Train compact robot controllers efficiently without costly simulations or manual reward design using the FIERCE learning framework.
- •For industrial robotics teams: Improve physical robot task performance by refining general policies into fast-adapting specialists using progress and failure feedback signals.
Authors
Runjia Tan, Yuang Tu, Yujie Yan, Lan Yu, Xuesong Tian, Chen Lv
Abstract
Generalist robot policies offer useful initialization, but refining compact specialists through limited physical interaction requires informative learning feedback. We present FIERCE, a generalist-initialized reinforcement learning framework centered on a unified, task-adaptive progress-failure evaluator. Its architecture shares an observation-language representation between an observed-progress head and an action-conditioned latent predictor whose past and current predictions feed a causal sequence head for task-failure estimation. Joint supervision from progress and preference labels, synchronized commands and observations, and terminal outcomes trains the evaluator; target-task rollouts support adaptation and calibration. Fixed evaluator snapshots provide progress shaping and failure-risk penalties alongside independently verified terminal rewards, while evaluator and policy updates alternate as new experience is collected. Refinement requires neither continued generalist action queries nor a dedicated target-task simulator or manually annotated dense rewards. Only the compact specialist is retained at deployment. The evaluation separates feedback quality, policy-learning efficiency, and deployment cost across simulation and two contact-rich real tasks. Code, model weights, and data-restoration tools are released at https://github.com/ar-mine/FIERCE.