Knowledge-Augmented Language Model and Its Application to Unsupervised Named-Entity Recognition (N19-1)
Copied to clipboard
| Challenge: | Current language models are unable to efficiently model entity names observed in text providing insufficient context. |
| Approach: | They propose to augment a traditional model with an external knowledge base to model entity names observed in text. |
| Outcome: | The proposed model improves on a Named Entity Recognition (NER) task by requiring no additional information such as named entity tags. |
Similar Papers
Distantly-Supervised Named Entity Recognition with Noise-Robust Learning and Language Model Augmented Self-Training (2021.emnlp-main)
Copied to clipboard
| Challenge: | Named entity recognition models require abundant high-quality annotations to train . distant supervision may induce incomplete and noisy labels, making supervised learning ineffective. |
| Approach: | They propose a noise-robust learning scheme for training named entity recognition models using only distantly-labeled data and a self-training method that uses contextualized augmentations created by pre-trained language models. |
| Outcome: | The proposed method outperforms existing supervised NER models on three datasets by significant margins. |
TRELM: Towards Robust and Efficient Pre-training for Knowledge-Enhanced Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for incorporating external knowledge into language models do not prioritize learning embeddings for entity-related tokens. |
| Approach: | They propose a framework for incorporating external knowledge into pre-training models that utilize entity-related tokens. |
| Outcome: | The proposed framework reduces pre-training time by 50% and outperforms other KEPLMs in knowledge probing tasks and multiple knowledge-aware language understanding tasks. |
KLMo: Knowledge Graph Enhanced Pretrained Language Model with Fine-Grained Relationships (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing knowledge-enhanced pretrained language models focus on entity information and ignore fine-grained relationships between entities. |
| Approach: | They propose to incorporate KG into the language learning process to obtain a KG-enhanced pretrained Language Model. |
| Outcome: | The proposed model improves on several knowledge-driven tasks, such as entity typing and relation classification, compared with the state-of-the-art knowledge-enhanced PLMs. |
Learning Knowledge-Enhanced Contextual Language Representations for Domain Natural Language Understanding (2023.emnlp-main)
Copied to clipboard
Taolin Zhang, Ruyao Xu, Chengyu Wang, Zhongjie Duan, Cen Chen, Minghui Qiu, Dawei Cheng, Xiaofeng He, Weining Qian
| Challenge: | Existing methods for pre-training KEPLMs with relational triples are difficult to adapt to close domains due to the lack of sufficient domain graph semantics. |
| Approach: | They propose a Knowledge-enhanced language representation learning framework for various closed domains that captures the implicit graph structure among the entities. |
| Outcome: | The proposed framework outperforms existing methods for pre-training KEPLMs in closed domains significantly. |
NERetrieve: Dataset for Next Generation Named Entity Recognition and Retrieval (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a widely adopted NLP task . authors present three variants of NER task, with dataset to support them . |
| Approach: | They propose three variants of the NER task, together with a dataset to support them . they propose a move towards more fine-grained entities and zero-shot recognition . |
| Outcome: | The proposed model matches or surpasses existing models in NER tasks . the proposed model is based on a large, silver-annotated corpus of 4 million paragraphs . |
What Matters for Neural Cross-Lingual Named Entity Recognition: An Empirical Analysis (D19-1)
Copied to clipboard
| Challenge: | Named entity recognition models are challenging for languages with little training data. |
| Approach: | They propose a simple and efficient neural architecture for cross-lingual named entity recognition models. |
| Outcome: | The proposed model achieves competitive performance with the state-of-the-art on two transferable factors: sequential order and multilingual embedding. |
Self-Adaptive Named Entity Recognition by Retrieving Unstructured Knowledge (2023.eacl-main)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is costly because of lack of training data and domain experts. |
| Approach: | They propose a self-adaptive neural model that retrieves external knowledge from unstructured text to learn the usages of entities that have not been learned well. |
| Outcome: | The proposed model outperforms strong baselines on cross-neuro-ner datasets by 2.35 points in F1 metric. |
KALA: Knowledge-Augmented Language Model Adaptation (2022.naacl-main)
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) have proved to be effective on various natural language understanding tasks. |
| Approach: | They propose a domain adaption framework which modulates the intermediate hidden representations of PLMs with domain knowledge, consisting of entities and their relational facts. |
| Outcome: | The proposed framework outperforms adaptive pre-training on question answering and named entity recognition tasks on multiple datasets across different domains. |
ACLM: A Selective-Denoising based Generative Data Augmentation Approach for Low-Resource Complex NER (2023.acl-long)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a task of detecting linguistically complex named entities in low-context text. |
| Approach: | They propose a keyword-based augmentation approach to address the context-entity mismatch issue in complex name recognition (NER) they use selective masking to retain the named entities and certain keywords in the input sentence that provide contextually relevant additional knowledge or hints about the named entity. |
| Outcome: | The proposed approach outperforms baseline methods on monolingual, cross-lingual, and multilingual complex NER in various low-resource settings. |
Named Entity Recognition Only from Word Embeddings (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing named entity recognition systems require large amounts of human annotated training data. |
| Approach: | They propose a fully unsupervised named entity recognition model which takes clues from pre-trained word embeddings. |
| Outcome: | The proposed model can be trained on two CoNLL benchmark datasets without annotating lexicon or corpus. |