SkillGrain improves personalized skill learning from user feedback
Learning What to Keep: Transfer-Aware Personal Skill Learning from User Feedback
Computation and Language
Summary
Personalized skills learned by language models can include both permanent user preferences and temporary corrections. The authors address the problem that simply keeping all feedback risks confusing the model with irrelevant changes. They introduce SkillGrain, a method that breaks down feedback into small rules and decides which ones to keep for future use. This helps the model better generalize across different tasks and requests, improving performance in personalized writing. Their experiments show SkillGrain works better than other approaches on multiple writing tasks.
What this means in practice
- •For software developers: Create AI writing assistants that adapt selectively to user preferences across different writing jobs.
- •For customer service teams: Improve automated responses by refining personalized skills based on ongoing user feedback without introducing harmful errors.
Authors
Li Wang, Chen Zhang, Yirong Yang, Xingyu Chen, Jinkun Zhao, Tianhao Peng, Wenjun Wu
Abstract
User feedback enables LLMs to refine reusable personalized skills, but often mixes persistent preferences with request-specific corrections. Indiscriminately retaining both risks negative transfer, motivating selective retention at the level of individual requirements. We frame this challenge as selective transfer over atomic rules and propose SkillGrain, a framework for learning reusable personalized skills from user feedback. SkillGrain decomposes feedback into independently actionable rules, predicts their transferability, and retains only those suitable for future requests. To learn these decisions, we combine rubric-guided skill rewards with fine-grained credit assignment that separately evaluates rule content and retention decisions. We further jointly optimize skill induction and skill-conditioned generation through reinforcement learning, ensuring that learned skills support downstream personalization. We construct a benchmark spanning five personalized writing tasks to evaluate cross-request generalization. Experiments across three base models in both in-domain and out-of-domain settings show that SkillGrain achieves the best overall average performance among competing methods.