Leveraging Expert Guided Adversarial Augmentation For Improving Generalization in Named Entity Recognition (2022.findings-acl)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) systems perform well on in-distribution data, but perform poorly on examples drawn from a shifted distribution. |
| Approach: | They propose to use expert-guided heuristics to change entity tokens and their contexts to alter their entity types as adversarial attacks. |
| Outcome: | The proposed model significantly improves performance on the challenging set and out-of-domain generalization. |
Similar Papers
AutoTriggER: Label-Efficient and Robust Named Entity Recognition with Auxiliary Trigger Extraction (2023.eacl-main)
Copied to clipboard
Dong-Ho Lee, Ravi Kiran Selvam, Sheikh Muhammad Sarwar, Bill Yuchen Lin, Fred Morstatter, Jay Pujara, Elizabeth Boschee, James Allan, Xiang Ren
| Challenge: | Named entity recognition models have shown impressive results in overcoming label scarcity and generalizing to unseen entities by leveraging distant supervision and auxiliary information such as explanations. |
| Approach: | They propose a framework that automatically generates and leverages “entity triggers” which are human-readable cues in the text that help guide the model to make better decisions. |
| Outcome: | The proposed framework outperforms the RoBERTa-CRF baseline by nearly 0.5 F1 points on three well-studied datasets. |
Context-aware Adversarial Training for Name Regularity Bias in Named Entity Recognition (2021.tacl-1)
Copied to clipboard
| Challenge: | Name Regularity Bias is a problem in NER models that use contextual information to predict the type of an ambiguous entity. |
| Approach: | They propose a model-agnostic training method that adds learnable adversarial noise to some entity mentions to improve their accuracy. |
| Outcome: | The proposed method outperforms feature-based models on name regularity bias . it adds learnable adversarial noise to some entity mentions, leading to gains . |
Guidance-Based Prompt Data Augmentation in Specialized Domains for Named Entity Recognition (2024.acl-short)
Copied to clipboard
| Challenge: | specialized fields such as science and biology face significant challenges due to the scarcity of quality data. |
| Approach: | They propose a guidance data augmentation technique that abstracts context and sentence structure and maintains context-entity relationships for DA. |
| Outcome: | The proposed method enhances the training performance of named entity recognition tasks while maintaining context-entity relationships. |
Towards Building More Robust NER datasets: An Empirical Study on NER Dataset Bias from a Dataset Difficulty View (2023.emnlp-main)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) models rely on superficial entity patterns for predictions, without considering evidence from the context. |
| Approach: | They propose to de-bias NER datasets by altering entity-context distribution . they also validate the feasibility of the proposed de-bianking techniques . |
| Outcome: | The proposed methods can be applied to different models and improve existing models. |
Improving Model Generalization: A Chinese Named Entity Recognition Case Study (2021.acl-short)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a fundamental building block for various downstream natural language processing tasks due to the ambiguous word boundaries and complex composition. |
| Approach: | They propose to resample entities within the same category to encourage a model to leverage both name and context knowledge in the training process. |
| Outcome: | The proposed method significantly improves a model’s ability to detect unseen entities, especially for company, organization and position categories. |
Data Augmentation for Cross-Domain Named Entity Recognition (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for named entity recognition focus on augmenting in-domain data in low-resource scenarios where annotated data is limited. |
| Approach: | They propose a neural architecture to transform data from high-resource to low-resourced domains by learning the patterns in the text that differentiate them. |
| Outcome: | The proposed approach improves on high-resource domain representations over high- and low-resourced domains. |
CoNLL#: Fine-grained Error Analysis and a Corrected Test Set for CoNLL-03 English (2024.lrec-main)
Copied to clipboard
| Challenge: | a glass ceiling for named entity recognition systems has been suggested for 2021 . however, the performance of the most popular NER benchmarks has plateaued since then . we investigate what NER models are still struggling with . |
| Approach: | They perform a fine-grained evaluation of the model outputs by adding document annotations to the CoNLL-03 English dataset to identify lingering errors. |
| Outcome: | The proposed model is able to correct errors and guide future work. |
Do CoNLL-2003 Named Entity Taggers Still Work Well in 2023? (2023.acl-long)
Copied to clipboard
| Challenge: | NER models trained on 20-year-old test set may not perform well on modern data. |
| Approach: | They evaluate the generalization of over 20 different models trained on the CoNLL-2003 dataset . they find no evidence of performance degradation in pre-trained Transformers . |
| Outcome: | The proposed model generalizations show that some models generalize well on new data while others do not. |
What do we really know about State of the Art NER? (2022.lrec-1)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a well researched task and widely used in real world NLP scenarios. |
| Approach: | They perform a broad evaluation of Named Entity Recognition using a popular dataset that takes into consideration various text genres and sources constituting the dataset at hand. |
| Outcome: | The proposed models perform on a popular dataset and generate six new adversarial test sets through small perturbations in the original test set, replacing select entities while retaining the context. |
ProgGen: Generating Named Entity Recognition Datasets Step-by-step with Self-Reflexive Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) exhibit remarkable adaptability across domains, but they are often not suitable for structured knowledge extraction tasks such as named entity recognition (NER). |
| Approach: | They propose a method that instructs LLMs to self-reflect on the specific domain and generates domain-relevant attributes for creating attribute-rich training data. |
| Outcome: | The proposed method produces NER datasets in domains with domain-relevant attributes and generates entity terms and NER context data around these entities. |