| Challenge: | Existing approaches to incremental sequence labeling have focused on the E2O and O2E issues, but neglect the O2e issue. |
| Approach: | They propose a framework for incremental sequence labeling without semantic shifts that mitigate catastrophic forgetting in models by using knowledge distillation to maintain the model’s discriminative ability for old entities. |
| Outcome: | The proposed framework mitigates catastrophic forgetting in models while maintaining discriminative ability for old entities while minimizing the model’s bias towards new entities. |
Similar Papers
Learning “O” Helps for Learning More: Handling the Unlabeled Entity Problem for Class-incremental NER (2023.acl-long)
Copied to clipboard
Ruotian Ma, Xuanting Chen, Zhang Lin, Xin Zhou, Junzhe Wang, Tao Gui, Qi Zhang, Xiang Gao, Yun Wen Chen
| Challenge: | Existing Named Entity Recognition systems are typically trained on a large-scale dataset with predefined entity classes, then deployed for entity recognition on the test data without further adaptation or refinement. |
| Approach: | They propose a representation learning method that adaptively detects entity clusters in "O" and two effective distance-based relabeling strategies for better learning the old classes. |
| Outcome: | The proposed method achieves 10.62% improvement over the baseline methods. |
Bringing Emerging Architectures to Sequence Labeling in NLP (2026.eacl-long)
Copied to clipboard
| Challenge: | Pretrained Transformer encoders are the dominant approach to sequence labeling . however, few have been applied to sequence labels on flat or simplified tasks . |
| Approach: | They propose to use pretrained Transformer encoders to model relations across words . they find that the architectures adapt well across tagging tasks that vary in complexity . |
| Outcome: | The proposed architectures perform well across tagging tasks across languages and datasets. |
Few-Shot Class-Incremental Learning for Named Entity Recognition (2022.acl-long)
Copied to clipboard
| Challenge: | Existing models of Named Entity Recognition (NER) are trained on large datasets with predefined entity classes, but data of new classes arrives constantly. Existing work on NER relies on the assumption that there exists abundance of labeled data for the training of new class. |
| Approach: | They propose a few-shot class-incremental learning problem where NER model is trained with only few labeled samples of the new classes without forgetting knowledge of the old ones. |
| Outcome: | The proposed model improves over existing baselines by reconstructing training data of old classes and real data from the training set. |
Beyond Boundaries: Learning a Universal Entity Taxonomy across Datasets and Languages for Open Named Entity Recognition (2025.coling-main)
Copied to clipboard
Yuming Yang, Wantong Zhao, Caishuang Huang, Junjie Ye, Xiao Wang, Huiyuan Zheng, Yang Nan, Yuran Wang, Xueying Xu, Kaixin Huang, Yunke Zhang, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Current Large Language Models struggle with complex entity taxonomies in open domains and lack NER capabilities. |
| Approach: | They propose a dataset to guide LLMs' generalization in Open NER under a universal entity taxonomy. |
| Outcome: | The proposed model outperforms GPT-4 in 3 out-of-domain benchmarks across 15 datasets and 6 languages. |
Continual Learning with Semi-supervised Contrastive Distillation for Incremental Neural Machine Translation (2024.acl-long)
Copied to clipboard
| Challenge: | Multi-domain learning is a good solution for solving domain tasks but it requires retraining when adding a new domain. |
| Approach: | They propose to exploit unlabeled data from the same distributions of the older domains to avoid catastrophic forgetting. |
| Outcome: | The proposed framework exploits unlabeled data from the same distributions of the older domains to avoid catastrophic forgetting. |
ChronosLex: Time-aware Incremental Training for Temporal Generalization of Legal Classification Tasks (2024.acl-long)
Copied to clipboard
| Challenge: | Existing models overlook the temporal dimension in their training process, leading to suboptimal performance over time. |
| Approach: | They propose a training paradigm that trains models on chronological splits, preserving the temporal order of the data. |
| Outcome: | The proposed model fails to fit to recent data, despite continual learning and temporal invariant methods. |
Context-aware Information-theoretic Causal De-biasing for Interactive Sequence Labeling (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing deep learning models for sequence labeling are expensive and time-consuming. |
| Approach: | They propose an interactive sequence labeling that allows training directly with the user feedback . they identify context and feedback biases by formulating interactive sequence labels via a Structural Causal Model. |
| Outcome: | The proposed approach can effectively alleviate the biases and can be learnt with the user feedback. |
New or Old? Exploring How Pre-Trained Language Models Represent Discourse Entities (2022.coling-1)
Copied to clipboard
| Challenge: | Recent research shows pre-trained language models learn to encode syntactic knowledge to a certain degree. |
| Approach: | They propose to investigate the information-status of entities as discourse-new or discourse-old . they use binary classification and sequence labeling to investigate their ability to encode syntactic knowledge . |
| Outcome: | The proposed models encode information on whether an entity has been introduced before or not in the discourse. |
OptSLA: an Optimization-Based Approach for Sequential Label Aggregation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to annotate data are based on binary and multi-choice problems, but little work has been done on complex tasks such as sequence labeling with imbalanced classes. |
| Approach: | They propose an optimization-based Sequential Label Aggregation method that integrates worker reliabilities and advanced deep learning techniques to conquer the challenge. |
| Outcome: | The proposed method outperforms state-of-the-art methods and is easier to interpret. |
Switch Point biased Self-Training: Re-purposing Pretrained Models for Code-Switching (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Code-switching (CS) is a phenomenon of switching between multiple languages . current models cannot handle CS due to lack of annotated data and limited resources. |
| Approach: | They propose a self-training method to repurpose existing models using a switch-point bias by leveraging unannotated data to reduce the gap between the switch point performance and retain overall performance on two distinct language pairs. |
| Outcome: | The proposed model reduces the gap between the switch point performance while retaining the overall performance on two distinct language pairs. |