Curriculum learning improved by reducing forgetting in math tasks
Towards Eliminating Catastrophic Forgetting in the Curriculum Learning of Math Reasoning Tasks
Machine Learning
Summary
When teaching a computer how to solve math problems step-by-step, it often forgets earlier tasks after learning harder ones. The paper shows that this forgetting happens because the computer’s learning settings shift between tasks and can’t be fully avoided. The authors propose a method called IV-EWC that helps the computer remember earlier tasks better by mixing in reminders during training. Tests show this method reduces forgetting and even improves performance on easier tasks after harder ones are learned.
What this means in practice
- •For machine learning engineers: Improve multi-task training pipelines for math problem solvers by integrating IV-EWC to reduce forgetting across curriculum stages.
- •For ai system designers: Design AI systems that maintain skills over sequential learning tasks, especially where task difficulty increases progressively.
Authors
Zengyan Yang, Yangyang Wu, Kai Huang, Pengfei Lyu, Tianyi Zhang, Mengying Zhu
Abstract
Curriculum learning has found broad application across numerous domains. Nevertheless, its effectiveness is intrinsically curtailed by catastrophic forgetting, driven by the shifts in model parameter distributions between curriculum tasks. In this paper, we investigate the phenomenon of catastrophic forgetting in this training paradigm, building on the established efficacy of curriculum learning. Our theoretical analyses of parameter update dynamics demonstrate that catastrophic forgetting in curriculum learning stems from the divergence of task optima, which is generally essential to the faster convergence of curriculum learning; therefore, forgetting cannot be completely eliminated. Based on this finding, we augment the training process and propose IV-EWC, which incorporates Elastic Weight Consolidation (EWC) into the curriculum learning objective to curb catastrophic forgetting in mathematical reasoning, a prototypical curriculum learning scenario. IV-EWC employs the influence function to construct a representative validation set from the curriculum's training data, which is used to drive dynamic regularization during training. We further present an extended theoretical analysis to show that EWC-based regularization methods mitigate catastrophic forgetting in curriculum learning, thereby providing theoretical support for IV-EWC. Empirical evaluations on three backbone models and three benchmarks indicate that curriculum learning exhibits catastrophic forgetting. IV-EWC alleviates this issue, reducing forgetting by 162% on average relative to vanilla curriculum learning and yielding positive backward transfer, as evidenced by improved performance on easier tasks after subsequent training on challenging tasks.