Learning Generalizable Skill Policy with Data-Efficient Unsupervised RL
2026-07-01 • Machine Learning
Machine LearningArtificial Intelligence
AI summaryⓘ
The authors study unsupervised reinforcement learning (URL), which trains agents without specific rewards to prepare them for later tasks. They find that existing methods struggle because the meanings of skills change over time and the agents don't generalize well to new situations. To fix this, the authors introduce GenDa, a new approach that relabels skills to keep their meaning stable and uses a technique called Complementary Information Bottleneck to make agents focus on important features. Their experiments show that GenDa learns more efficiently and works better when applied to new tasks.
unsupervised reinforcement learningskill-conditioned policiesoff-policy methodsskill relabelingnon-stationaritygeneralizationComplementary Information Bottleneckdata efficiencyego-centric featuresdistribution shifts
Authors
Jongchan Park, Seungjun Oh, Seungho Baek, Yusung Kim
Abstract
Unsupervised Reinforcement Learning (URL) aims to pre-train scalable, skill-conditioned policies without extrinsic rewards, serving as a foundation for downstream control tasks. Despite recent progress, we argue that current off-policy URL methods are limited by two critical, overlooked bottlenecks: (1) non-stationary skill semantics and (2) brittle generalization. To address these challenges, we propose GenDa (Generalizable Data-efficient Agent), a unified framework for robust unsupervised reinforcement learning. First, we introduce a skill relabeling mechanism to mitigate non-stationarity and significantly improve data efficiency for pre-training. Second, we propose a Complementary Information Bottleneck (CIB), encouraging the learned skill policy to focus on ego-centric features and become robust to distribution shifts for downstream tasks. Through various experiments, we demonstrate that GenDa significantly enhances the scalability of URL with superior generalizability and data efficiency. Our code and videos are available at https://ihatebroccoli.github.io/official-GenDa.