Hierarchical method improves continual learning for zero-shot graph models
Hierarchical Response Preservation for Continual Adaptation of Zero-Shot Graph-Text Models
Machine Learning
Summary
Graphs and text can be linked by models to recognize new categories without prior examples, but as new classes appear, these models can forget earlier knowledge or fail to learn new classes well. The authors propose a hierarchical way to keep old class information while still learning new classes effectively. This method helps models keep good performance on old tasks and still adapt to new data. Tests show this approach improves accuracy and maintains zero-shot capabilities across various settings.
What this means in practice
- •For machine learning engineers: Improve continual adaptation of graph-text models in systems needing recognition of evolving classes without retraining from scratch.
- •For data platform teams: Maintain zero-shot transfer accuracy in graph data platforms handling incremental class updates over time.
Authors
Haopeng Zhang, Yuhan Wang, Yubing Su, Yingxin Chen, Xiao Wang, Ruijie Wang, Jianxin Li
Abstract
Pretrained graph-text models align graph representations with textual semantics, enabling recognition of unseen classes and transfer across graph domains. However, as graph data and classes continually arrive, models should learn from new supervision while retaining their zero-shot transfer capabilities and historical task knowledge. Two challenges arise: (i) new classes can overturn historical predictions despite preserved distinctions among historical classes, and (ii) overly strict response preservation can stall learning of new classes. To address these challenges, we propose Hierarchical Response Preservation (HiRP). HiRP represents this competition through a hierarchical response that keeps each historical-class probability and sums new-class probabilities, preserving historical distinctions and aggregate competition while allowing distinctions within the new class group to adapt. It further uses the geometry induced by this response to guide constrained updates, retaining useful adaptation directions while controlling response drift. Across three class-incremental settings, HiRP achieves absolute gains of 1.84-7.95 percentage points in average accuracy over the strongest compared baseline in each setting, while mitigating zero-shot transfer degradation.