Contextualized Word Representations from Distant Supervision with and for NER (D19-55)
Copied to clipboard
| Challenge: | Existing word embeddings for named entity recognition are stacked with traditional ones for downstream tasks. |
| Approach: | They propose a special type of contextualized word representation that is learned from distant supervision annotations and dedicated to named entity recognition. |
| Outcome: | The proposed representation surpasses the existing representations and is complementary to existing embeddings. |
Similar Papers
Improved Word Sense Disambiguation Using Pre-Trained Contextualized Word Representations (D19-1)
Copied to clipboard
| Challenge: | Contextualized word representations are effective in downstream tasks such as question answering, named entity recognition, and sentiment analysis. |
| Approach: | They propose to integrate pre-trained contextualized word representations into a neural network that captures the whole sentence and the word representation in the sentence. |
| Outcome: | The proposed approach outperforms the state-of-the-art approach that makes use of non-contextualized word embeddings on multiple benchmark WSD datasets. |
Distant Supervision from Disparate Sources for Low-Resource Part-of-Speech Tagging (D18-1)
Copied to clipboard
| Challenge: | Low-resource languages lack manual annotated data to learn basic models such as part-of-speech (POS) taggers. |
| Approach: | They propose a cross-lingual neural part-of-speech tagger that learns from disparate sources of distant supervision in a uniform framework. |
| Outcome: | The proposed model scales to hundreds of low-resource languages without access to gold annotated data. |
Distantly-Supervised Named Entity Recognition with Noise-Robust Learning and Language Model Augmented Self-Training (2021.emnlp-main)
Copied to clipboard
| Challenge: | Named entity recognition models require abundant high-quality annotations to train . distant supervision may induce incomplete and noisy labels, making supervised learning ineffective. |
| Approach: | They propose a noise-robust learning scheme for training named entity recognition models using only distantly-labeled data and a self-training method that uses contextualized augmentations created by pre-trained language models. |
| Outcome: | The proposed method outperforms existing supervised NER models on three datasets by significant margins. |
How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings (D19-1)
Copied to clipboard
| Challenge: | Existing word embeddings were static, requiring all senses of a polysemous word to share the same representation. |
| Approach: | They found that the contextualized representations of all words are not isotropic in any layer of the contextualizing model. |
| Outcome: | The results show that the representations of all words are not isotropic in any layer of the contextualizing model. |
Pooled Contextualized Embeddings for Named Entity Recognition (N19-1)
Copied to clipboard
| Challenge: | Contextual string embeddings are a recent type of word embeddable that are useful for sequence labeling tasks. |
| Approach: | They propose a method that dynamically aggregates contextualized embeddings of each unique string . they then use a pooling operation to distill a ”global” word representation from all contextualized instances . |
| Outcome: | The proposed method improves state-of-the-art for named entity recognition tasks. |
Distantly Supervised Named Entity Recognition using Positive-Unlabeled Learning (P19-1)
Copied to clipboard
| Challenge: | Empirical studies on four public NER datasets demonstrate the effectiveness of our proposed method. |
| Approach: | They propose a method to perform named entity recognition using unlabeled data and named entity dictionaries. |
| Outcome: | The proposed method can estimate task loss as if there is fully labeled data. |
Deep Contextualized Word Representations (N18-1)
Copied to clipboard
Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, Luke Zettlemoyer
| Challenge: | a new type of deep contextualized word representation is proposed for language understanding problems . word vectors are learned functions of the internal states of a deep bidirectional language model . |
| Approach: | They propose a new type of deep contextualized word representation that models complex features of word use and how they vary across linguistic contexts. |
| Outcome: | The proposed representations improve the state of the art across six challenging NLP problems. |
Unsupervised Cross-Lingual Representation Learning (P19-4)
Copied to clipboard
| Challenge: | a comprehensive survey of cutting-edge weakly-supervised and unsupervised cross-lingual word representations is presented . |
| Approach: | This tutorial provides a comprehensive survey of recent work on weakly-supervised and unsupervised cross-lingual word representations. |
| Outcome: | This tutorial provides a comprehensive survey of cutting-edge weakly-supervised and unsupervised word representations. |
Advances in Pre-Training Distributed Word Representations (L18-1)
Copied to clipboard
| Challenge: | Pre-trained word representations are a building block of many Natural Language Processing and Machine Learning applications. |
| Approach: | They propose to combine known tricks and a set of publicly available pre-trained word vector representations to train high-quality representations. |
| Outcome: | The proposed models outperform the current state of the art on a number of tasks while maintaining a high training speed to scale to massive amount of data. |
Learning Named Entity Tagger using Domain-Specific Dictionary (D18-1)
Copied to clipboard
| Challenge: | Existing methods to build reliable named entity recognition systems require large amounts of manually-labeled training data. |
| Approach: | They propose a revised fuzzy CRF layer to handle tokens with multiple possible labels to address noisy distant supervision. |
| Outcome: | The proposed model can handle tokens with multiple possible labels under the traditional framework and improves on the existing model with a new Tie or Break scheme. |