Probing Pre-trained Auto-regressive Language Models for Named Entity Typing and Recognition (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing studies have focused on auto-regressive models for generalization in named entity (NE) typing (NET) and recognition (NER) . however, little has been done in this direction for auto-Regressive LMs despite their popularity and potential to express a wide variety of NLP tasks in the same unified format. |
| Approach: | They propose to probe auto-regressive LMs for NET and NER generalization by resorting to meta-learning to assess the model's memorization of NEs. |
| Outcome: | The proposed model performs well on NET and NER generalization tasks, while relying more on NE than contextual cues in few-shot NER. |
Similar Papers
Memorisation versus Generalisation in Pre-trained Language Models (2022.acl-long)
Copied to clipboard
| Challenge: | State-of-the-art pre-trained language models have been shown to memorise facts and perform well with limited amounts of training data. |
| Approach: | They propose to extend pre-trained language models to generalise and memorise facts in noisy and low-resource scenarios. |
| Outcome: | The proposed extension improves performance in low-resource named entity recognition tasks. |
NERetrieve: Dataset for Next Generation Named Entity Recognition and Retrieval (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a widely adopted NLP task . authors present three variants of NER task, with dataset to support them . |
| Approach: | They propose three variants of the NER task, together with a dataset to support them . they propose a move towards more fine-grained entities and zero-shot recognition . |
| Outcome: | The proposed model matches or surpasses existing models in NER tasks . the proposed model is based on a large, silver-annotated corpus of 4 million paragraphs . |
Coarse-to-Fine Pre-training for Named Entity Recognition (2020.emnlp-main)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a task of discovering information entities and identifying their corresponding categories. |
| Approach: | They propose a NER-specific framework to inject coarse-to-fine named entity knowledge into pre-trained models by using a remote supervision strategy. |
| Outcome: | The proposed framework achieves significant improvements against several pre-trained base-lines, demonstrating its effectiveness in label-few and low-resource scenarios. |
Distantly-Supervised Named Entity Recognition with Noise-Robust Learning and Language Model Augmented Self-Training (2021.emnlp-main)
Copied to clipboard
| Challenge: | Named entity recognition models require abundant high-quality annotations to train . distant supervision may induce incomplete and noisy labels, making supervised learning ineffective. |
| Approach: | They propose a noise-robust learning scheme for training named entity recognition models using only distantly-labeled data and a self-training method that uses contextualized augmentations created by pre-trained language models. |
| Outcome: | The proposed method outperforms existing supervised NER models on three datasets by significant margins. |
NuNER: Entity Recognition Encoder Pre-training via LLM-Annotated Data (2024.emnlp-main)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a core component of natural language processing, present in a variety of applications such as medical coding, financial news analysis, or legal documents parsing. |
| Approach: | They propose to use Large Language Models (LLMs) to create NuNER, a compact language representation model specialized in the Named Entity Recognition task. |
| Outcome: | The proposed model outperforms similar-sized foundation models in the few-shot regime and is based on a human-annotated dataset. |
Few-Shot Named Entity Recognition: An Empirical Baseline Study (2021.emnlp-main)
Copied to clipboard
Jiaxin Huang, Chunyuan Li, Krishan Subudhi, Damien Jose, Shobana Balakrishnan, Weizhu Chen, Baolin Peng, Jianfeng Gao, Jiawei Han
| Challenge: | Existing methods to build named entity recognition systems with limited labeled data are lacking. |
| Approach: | They propose three orthogonal schemes to build named entity recognition systems when labeled data is limited. |
| Outcome: | The proposed NER systems outperform existing methods on few-shot and training-free settings. |
GPT-NER: Named Entity Recognition via Large Language Models (2025.findings-naacl)
Copied to clipboard
Shuhe Wang, Xiaofei Sun, Xiaoya Li, Rongbin Ouyang, Fei Wu, Tianwei Zhang, Jiwei Li, Guoyin Wang, Chen Guo
| Challenge: | Large-scale language models (LLMs) have shown impressive ability for in-context learning with limited training data. |
| Approach: | They propose a novel sequence labeling task that transforms a sequence labeled as a text-generation task into a self-verification task that LLMs can adapt to. |
| Outcome: | The proposed model performs better on NER than supervised models on a variety of tasks . the proposed model can be easily adapted by LLMs to generate a text sequence . |
Formulating Few-shot Fine-tuning Towards Language Model Pre-training: A Pilot Study on Named Entity Recognition (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning pre-trained language models are limited . we propose a few-shot fine-uning framework for NER . |
| Approach: | They propose a few-shot fine-tuning framework for named entity recognition (NER) they propose three new types of tokens, "is-entity", "which-type" and "bracket" |
| Outcome: | The proposed framework improves on pre-trained language models on several benchmark datasets. |
Self-Improving for Zero-Shot Named Entity Recognition with Large Language Models (2024.naacl-short)
Copied to clipboard
| Challenge: | Existing studies exploring the performance of large language models on named entity recognition tasks have focused on training task-specific LLMs for NER. |
| Approach: | They propose a training-free self-improving framework that utilizes an unlabeled corpus to stimulate the self-learning ability of LLMs. |
| Outcome: | The proposed framework improves performance on the named entity recognition task by using an unlabeled corpus. |
Exploring Nested Named Entity Recognition with Large Language Models: Methods, Challenges, and Insights (2024.emnlp-main)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is a challenging task in natural language processing . nested NER requires sophisticated techniques to identify entities within entities . |
| Approach: | They investigate the application of Large Language Models (LLMs) to nested NER . they find methodologies from previous work are less effective . |
| Outcome: | The proposed methods outperform BERT-based models in nested NER tasks . however, they do not outperformed the existing models on the GENIA dataset . |