Evolving functional repertoires improve continual learning in large language models
From a Prompt to Repertoires: Evolving Functional REpertoires Enable LLM Continual Learning
Machine LearningComputation and Language
Summary
Large language models struggle to learn new tasks continuously without forgetting what they already know. The authors found that simply optimizing prompts for each new task causes the model to forget previous skills because the prompts overfit to new tasks. They propose a new method called Evolving Functional REpertoires (EFRE), which maintains a collection of prompts that adapt and grow as new tasks come in. This approach helps the model keep old knowledge while learning new tasks better than previous methods.
What this means in practice
- •For natural language processing engineers: Maintain and improve language model skills on sequential tasks without retraining or extensive parameter updates.
- •For ai system integrators: Build AI agents that adapt continually to new tasks while preserving performance on previous ones.
Authors
Fengyuan Liu, Yue Wang, Hangxi Guo, Fengyuan Liu, Chenxu Wu, Yanguang Liu, Mengnan Du
Abstract
Continual learning remains challenging for large language models, which must enable models to acquire new skills and knowledge without degrading existing capabilities. Existing approaches typically address this challenge by carefully designing how model parameters are updated. In contrast, prompt optimization avoids costly parameter updates while achieving competitive or even superior performance to reinforcement learning methods such as GRPO on individual knowledge-intensive and reasoning tasks. This raises a natural question: \textit{Can prompt optimization, as an efficient adaptation approach, be directly applied to continual learning?} Our analysis shows that, under sequential task adaptation, it suffers from catastrophic forgetting, while optimized prompts accumulate rules that overfit to local task distributions. To address these limitations, we propose \emph{Evolving Functional REpertoires} (EFRE), which replaces a single prompt with a repertoire of functions that evolves as new tasks arrive: compatible updates refine existing functions, while conflicting updates trigger the emergence of new ones. On a three-task continual-learning stream, EFRE achieves a final average performance 7.50 percentage points higher than GRPO. Moreover, after adaptation to the Bio task, its performance on FinQA decreases by only 1.56 percentage points, compared with 25.10 percentage points for the base prompt optimization method. We further instantiate EFRE in a minimal agent system and observe consistent improvements across different backbone models. Overall, these results demonstrate EFRE's strong performance in continual learning for large language models and highlight its substantial potential for continual learning in advanced agent systems.