KuaiRP role-playing models balance knowledge and general ability efficiently
KuaiRP Series Role-playing Models Technical Report
Artificial IntelligenceComputation and Language
Summary
Role-playing AI models often struggle to combine deep knowledge about specific characters with general abilities to respond well overall. The authors designed a new training process that first teaches the model using detailed character templates and rules, then uses reinforcement learning and a special technique called self-distillation to keep the model’s broad skills intact. This approach helps create smaller, stable models that act closely like real role-players while still giving good general responses. Their experiments show the models perform well without needing big computing resources.
What this means in practice
- •For chatbot developers: Build conversational agents that remember detailed character traits yet respond flexibly in varied scenarios using the KuaiRP training method.
- •For game developers: Deploy efficient AI-driven NPCs that act with consistent personality and domain knowledge without heavy computational cost in role-playing games.
Authors
Yipeng Wang, Ziwei Zhang, Jiahui Zhang, Qi Gan, Kai Sheng
Abstract
This paper introduces the complete technical solution for the KuaiRP series of role-playing models. We aim to achieve four core objectives for a dedicated role-playing model: simplified prompt engineering, highly stable output quality, built-in domain world knowledge, and high-efficiency deployment with a small parameter size. However, effectively injecting deep domain knowledge often leads to a severe catastrophic forgetting of the model's general agent capabilities. To overcome this trade-off, we propose a multi-stage training pipeline. First, we design a standardized character template and construct an SFT data pipeline based on user behavior simulation and reverse profile filtering. Next, we utilize a rule-based composite reward function during the Reinforcement Learning (RL) phase to eliminate common degradation phenomena like length expansion and repetitive generation. Finally, to recover the general capabilities compromised during SFT and RL, we propose a novel self-distillation paradigm using Two-stage On-Policy Distillation (OPD) equipped with Cumulative-Divergence Decay (CDD). By using the domain-adapted model as the teacher and the original base model as the student, we effectively balance deep domain knowledge injection with the preservation of general agent capabilities. Experimental results demonstrate that the KuaiRP models not only match the current state-of-the-art proprietary models in role-playing fidelity within our target domains, but also successfully recover general agent capabilities, maintaining extremely low deployment costs.