Do “English” Named Entity Recognizers Work Well on Global Englishes? (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Most of English named entity recognition datasets contain American or British English data . multiple problems may occur in low-resource English contexts, such as confusion of named entities with regionspecific meanings . |
| Approach: | They build a newswire dataset to analyze NER model performance on low-resource English variants . they find that models trained on the CoNLL or OntoNotes datasets experienced significant performance drops . |
| Outcome: | The results show that models trained on the CoNLL or OntoNotes datasets experienced significant performance drops. |
Similar Papers
CoNLL#: Fine-grained Error Analysis and a Corrected Test Set for CoNLL-03 English (2024.lrec-main)
Copied to clipboard
| Challenge: | a glass ceiling for named entity recognition systems has been suggested for 2021 . however, the performance of the most popular NER benchmarks has plateaued since then . we investigate what NER models are still struggling with . |
| Approach: | They perform a fine-grained evaluation of the model outputs by adding document annotations to the CoNLL-03 English dataset to identify lingering errors. |
| Outcome: | The proposed model is able to correct errors and guide future work. |
Do CoNLL-2003 Named Entity Taggers Still Work Well in 2023? (2023.acl-long)
Copied to clipboard
| Challenge: | NER models trained on 20-year-old test set may not perform well on modern data. |
| Approach: | They evaluate the generalization of over 20 different models trained on the CoNLL-2003 dataset . they find no evidence of performance degradation in pre-trained Transformers . |
| Outcome: | The proposed model generalizations show that some models generalize well on new data while others do not. |
Comparing Annotated Datasets for Named Entity Recognition in English Literature (2022.lrec-1)
Copied to clipboard
| Challenge: | Generally speaking, the majority of NER tools struggle to perform well when the entities in the text contain specific characteristics. |
| Approach: | They analysed two existing annotated datasets and two additional gold standard datasets to evaluate the performance of two NER tools. |
| Outcome: | The results show that the performance of two NER tools varies significantly depending on the gold standard used for the individual evaluations. |
The Role of Global and Local Context in Named Entity Recognition (2023.acl-short)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) models are usually applied sequentially because of their complexity. |
| Approach: | They explore the impact of global document context on Named Entity Recognition . they find that correctly retrieving global document contextual has a greater impact . |
| Outcome: | The proposed model can retrieve global context better than leveraging local context . authors say the model can push the state of the art further . |
DynamicNER: A Dynamic, Multilingual, and Fine-Grained Dataset for LLM-based Named Entity Recognition (2025.emnlp-main)
Copied to clipboard
Hanjun Luo, Yingbin Jin, Yiran Wang, Xinfeng Li, Tong Shang, Xuecheng Liu, Ruizhe Chen, Kun Wang, Hanan Salam, Qingsong Wen, Zuozhu Liu
| Challenge: | Existing datasets designed for Named Entity Recognition methods are inadequate for LLMs. |
| Approach: | They propose a dataset that is multilingual and multi-granular and enables LLMs to be applied to Named Entity Recognition methods. |
| Outcome: | The proposed dataset is multilingual and multi-granular, covering 8 languages and 155 entity types, with corpora spanning a diverse range of domains. |
CleanCoNLL: A Nearly Noise-Free Named Entity Recognition Dataset (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing models achieve F1-scores comparable to or exceed noise level in CoNLL-03 . current models have significant annotation errors, incompleteness, and inconsistencies in the data . |
| Approach: | They propose to add a layer of entity linking annotation to the CoNLL-03 corpus to correct 7.0% of all labels. |
| Outcome: | The proposed approach corrects 7.0% of all labels in the English CoNLL-03 dataset. |
Universal NER: A Gold-Standard Multilingual Named Entity Recognition Benchmark (2024.naacl-long)
Copied to clipboard
Stephen Mayhew, Terra Blevins, Shuheng Liu, Marek Suppa, Hila Gonen, Joseph Marvin Imperial, Börje Karlsson, Peiqin Lin, Nikola Ljubešić, Lester James Miranda, Barbara Plank, Arij Riabi, Yuval Pinter
| Challenge: | In named entity recognition, the majority of annotation efforts are centered on English, and cross-lingual transfer performance remains brittle. |
| Approach: | They propose to develop gold-standard named entity recognition benchmarks in many languages using a cross-lingual consistent schema. |
| Outcome: | The proposed benchmarks will be released to the public in 2022 . they will provide baselines on in-language and cross-lingual learning settings. |
Evaluation of Off-the-shelf Speech Recognizers on Different Accents in a Dialogue Domain (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing automatic speech recognition systems for non-American accents have a much higher error rate than for general american accents. |
| Approach: | They evaluate automatic speech recognition systems on agent-directed speech . they find that the performance is worse for non-American accents than for General American . |
| Outcome: | The ASR systems perform worse for non-American accents than for General American accents . the results suggest that training on non-native English speakers is needed to narrow the performance gap. |
NNE: A Dataset for Nested Named Entity Recognition in English Newswire (P19-1)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is widely used in downstream tasks but most tools focus on flat mention structure over coarse schemas. |
| Approach: | They describe a fine-grained, nested named entity dataset over the Wall Street Journal portion of the Penn Treebank. |
| Outcome: | The proposed dataset comprises 279,795 mentions of 114 entity types with up to 6 layers of nesting. |
OpenNER 1.0: Standardized Open-Access Named Entity Recognition Datasets in 50+ Languages (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing datasets are not consistently formatted and use a variety of chunk encodings (IOB, BIO, etc.), often without documentation. |
| Approach: | They present OpenNER 1.0, a standardized collection of openly-available named entity recognition (NER) datasets. |
| Outcome: | The proposed datasets correct annotation format issues and provide a structure that enables research in multilingual and multi-ontology NER. |