Type-Balanced Contextual Learning for Incremental Named Entity Recognition
2026-08-31 • Computation and Language
Computation and Language
AI summaryⓘ
The authors study a task called Incremental Named Entity Recognition (INER), which is about finding new types of entities in text over time. They find that old entities often get confused with new ones because the context—the words around them—changes in biased ways, making it harder to remember old information. To fix this, they create a new learning method called Type-Balanced Contextual Learning (TBCL) that balances context information and keeps learning stable. Their experiments show that TBCL helps INER systems remember old entities better while learning new ones.
Incremental Named Entity RecognitionNamed Entity RecognitionCatastrophic ForgettingPseudo-labelingContextual LearningContext BiasSentence-duplet LearningContextual Consistency LossInformation Extraction
Authors
Duzhen Zhang, Yahan Yu, Xiuyi Chen, Chenxing Li, Dong Yu
Abstract
Incremental Named Entity Recognition (INER) stands as a pivotal task in information extraction, emphasizing the successive identification of new entity types within unstructured text. Faced with the continuous influx of entity types, INER grapples with two significant challenges: the widespread issue of catastrophic forgetting and the unique shift issue of the non-entity type semantics. While pseudo-labeling-based INER methods have proven effective in addressing these challenges, a previously overlooked issue arises: the biased context problem. Our analysis shows that, in new sentences, the contextual associations of tokens representing old entity types exhibit a significantly stronger bias towards new entity types compared to their contexts in old sentences. This tendency intensifies the degradation of old knowledge while promoting the overfitting of new knowledge. To solve this biased context, we propose a Type-Balanced Contextual Learning (TBCL) method, featuring a sentence-duplet learning scheme and a contextual consistency loss. This approach offers a fresh perspective for INER through context analysis. Extensive experiments across ten INER settings on three highly recognized datasets showcase the efficacy of our TBCL method, highlighting its proficiency in resolving the biased context issue inherent in pseudo-labeling based INER approaches.