| Challenge: | Disambiguated skip-gram is a neural-probabilistic model for learning multi-sense word embeddings. |
| Approach: | They propose a model that is end-to-end differentiable and can be interpreted as a feed-forward neural network. |
| Outcome: | The proposed model improves state-of-the-art in word sense induction benchmarks. |
Similar Papers
Embedding Words as Distributions with a Bayesian Skip-gram Model (C18-1)
Copied to clipboard
| Challenge: | Rather than assuming that word embeddings are fixed across the entire text collection, we generate them from word-specific prior densities for each word. |
| Approach: | They propose a method for embedding words as probability densities in a low-dimensional space from a word-specific prior density for each occurrence of a given word. |
| Outcome: | The proposed method can encode word as a distribution on a range of benchmarks and is comparable to Gaussian embeddings. |
Directional Skip-Gram: Explicitly Distinguishing Left and Right Context for Word Embeddings (N18-2)
Copied to clipboard
| Challenge: | Existing word embedding models are limited by semantic resources, which are hard to obtain or annotate. |
| Approach: | They propose a directional skip-gram model that explicitly distinguishes between left and right contexts in word prediction. |
| Outcome: | The proposed model outperforms other models on different datasets in semantic and syntactic evaluations. |
Language Modelling Makes Sense: Propagating Representations through WordNet for Full-Coverage Word Sense Disambiguation (P19-1)
Copied to clipboard
| Challenge: | Contextual embeddings address the problem of meaning conflation hampering word embeddables. |
| Approach: | They propose a method that creates sense-level embeddings with full-coverage of WordNet without recourse to explicit sense distributions or task-specific modelling. |
| Outcome: | The proposed method surpasses previous systems using powerful models and is robust when ignoring part-of-speech and lemma features. |
Probabilistic FastText for Multi-Sense Word Embeddings (P18-1)
Copied to clipboard
| Challenge: | Probabilistic FastText model for word embeddings captures word senses, sub-word structure, and uncertainty information. |
| Approach: | They propose a model for word embeddings that captures multiple word senses . they represent each word with a Gaussian mixture density, with each vector representing an n-gram . |
| Outcome: | The proposed model outperforms dictionary-level probabilistic embeddings on word-similarity benchmarks. |
Word Sense Disambiguation for 158 Languages using Word Embeddings Only (2020.lrec-1)
Copied to clipboard
Varvara Logacheva, Denis Teslenko, Artem Shelmanov, Steffen Remus, Dmitry Ustalov, Andrey Kutuzov, Ekaterina Artemova, Chris Biemann, Simone Paolo Ponzetto, Alexander Panchenko
| Challenge: | Existing methods of disambiguation of word senses are based on knowledge bases, taxonomies, and other externally built resources. |
| Approach: | They propose a method that takes a pre-trained word embedding model and induces a fully-fledged word sense inventory for 158 languages. |
| Outcome: | The proposed model is based on a pre-trained word embedding model and induces a fully-fledged word sense inventory in 158 languages. |
Improved Word Sense Disambiguation Using Pre-Trained Contextualized Word Representations (D19-1)
Copied to clipboard
| Challenge: | Contextualized word representations are effective in downstream tasks such as question answering, named entity recognition, and sentiment analysis. |
| Approach: | They propose to integrate pre-trained contextualized word representations into a neural network that captures the whole sentence and the word representation in the sentence. |
| Outcome: | The proposed approach outperforms the state-of-the-art approach that makes use of non-contextualized word embeddings on multiple benchmark WSD datasets. |
One Classifier for All Ambiguous Words: Overcoming Data Sparsity by Utilizing Sense Correlations Across Words (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing word-specific classifiers lack the ability to generalize across words and require limited sense-annotated data for every word. |
| Approach: | They propose to learn a single model that derives sense representations and enforces congruence between a word instance and its right sense by using both sense-annotated data and lexical resources. |
| Outcome: | Empirical evaluation shows the proposed model outperforms classifier-based models by 1.7%, 2.5% and 3.8% in F1-score on GloVe, ELMo and BERT word embeddings respectively. |
Moving Down the Long Tail of Word Sense Disambiguation with Gloss Informed Bi-encoders (2020.acl-main)
Copied to clipboard
| Challenge: | Existing models for Word Sense Disambiguation are not uniformly distributed on rare or unseen senses. |
| Approach: | They propose a bi-encoder model that embeds the target word with its context and the dictionary definition, or gloss, of each sense. |
| Outcome: | The proposed model outperforms previous state-of-the-art models on English all-words WSD, with a 31.1% error reduction on less frequent senses over prior work. |
Detecting Word Sense Disambiguation Biases in Machine Translation for Model-Agnostic Adversarial Attacks (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown that neural machine translation models rely heavily on source sentence information when resolving lexical ambiguity. |
| Approach: | They propose a method for the prediction of disambiguation errors based on statistical data properties and propose 'a simple adversarial attack strategy' that minimally perturbs sentences to elicit disambiguations errors. |
| Outcome: | The proposed method shows that disambiguation robustness varies substantially between domains and different models trained on the same data are vulnerable to different attacks. |
A Probabilistic Model for Joint Learning of Word Embeddings from Texts and Images (D18-1)
Copied to clipboard
| Challenge: | Existing approaches combine language and perception to infer word embeddings . however, the embeddables produced by such models do not reflect the actual word representations. |
| Approach: | They propose a probabilistic model that integrates linguistic and perceptual inputs to explain observed word-context pairs in a text corpus. |
| Outcome: | The proposed model achieves competitive or stronger results on tasks of assessing pairwise word similarity and image/caption retrieval compared to other state-of-the-art models. |