| Challenge: | Existing methods for learning lower-dimensional representations of words using unlabelled data learn a single representation for a word, ignoring the different senses of that word (polysemy). |
| Approach: | They propose a method that jointly learns sense-aware word embeddings using both unlabelled and sense-tagged text corpora. |
| Outcome: | The proposed method outperforms competing methods on word similarity and short-text classification benchmark datasets. |
Similar Papers
Together We Make Sense–Learning Meta-Sense Embeddings (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing sense embeddings do not cover all senses of ambiguous words equally well due to discrepancies in their training resources. |
| Approach: | They propose a meta-sense embedding method that preserves sense neighbourhoods by combining multiple independently trained source sense embeddables. |
| Outcome: | The proposed method outperforms several baselines on Word Sense Disambiguation and Word-in-Context tasks. |
Joint Embedding of Words and Labels for Text Classification (P18-1)
Copied to clipboard
Guoyin Wang, Chunyuan Li, Wenlin Wang, Yizhe Zhang, Dinghan Shen, Xinyuan Zhang, Ricardo Henao, Lawrence Carin
| Challenge: | Existing approaches to text classification use word embeddings to capture semantic regularities between words. |
| Approach: | They propose to view text classification as a label-word joint embedding problem . they use a framework that measures compatibility between text sequences and labels . |
| Outcome: | The proposed framework outperforms the state-of-the-art methods on large text datasets. |
Constructing High Quality Sense-specific Corpus and Word Embedding via Unsupervised Elimination of Pseudo Multi-sense (L18-1)
Copied to clipboard
| Challenge: | Existing word embedding frameworks distinguish different senses of words by their contexts. |
| Approach: | They propose a framework for unsupervised corpus sense tagging which trains multi-sense word embeddings on a given corpus. |
| Outcome: | The proposed framework detects pseudo multi-senses without extra language resources without additional language resources. |
Unsupervised Joint Training of Bilingual Word Embeddings (P19-1)
Copied to clipboard
| Challenge: | Existing methods for unsupervised bilingual word embeddings are limited by the dissimilarity between the word embedded spaces. |
| Approach: | They propose a method that trains unsupervised bilingual word embeddings jointly on parallel data generated through unsupervised machine translation. |
| Outcome: | The proposed method outperforms unsupervised mapped bilingual word embeddings in cross-lingual NLP tasks. |
A Simple Approach to Learning Unsupervised Multilingual Embeddings (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work on unsupervised cross-lingual embeddings in the bilingual setting has given the impetus to learning a shared embeddable space for several languages. |
| Approach: | They propose to solve two sub-problems together to learn a shared embedding space for several languages. |
| Outcome: | The proposed approach outperforms existing methods in bilingual lexicon induction, cross-lingual word similarity, multilingual document classification, and multilingual dependency parsing tasks. |
Learning Domain-Sensitive and Sentiment-Aware Word Embeddings (P18-1)
Copied to clipboard
| Challenge: | Existing word embeddings cannot produce domain-sensitive embeddables due to domain-specific nature of words. |
| Approach: | They propose a method for learning domain-sensitive and sentiment-aware embeddings that captures sentiment semantics and domain sensitivity of individual words. |
| Outcome: | The proposed method can produce domain-common embeddings and domain-specific embedds. |
LSTMEmbed: Learning Word and Sense Representations from a Large Semantically Annotated Corpus with Long Short-Term Memories (P19-1)
Copied to clipboard
| Challenge: | Recent work has focused on vector representations which capture different meanings, i.e., senses, of words. |
| Approach: | They propose a bidirectional LSTM model which learns word senses from semantically annotated corpora by focusing on word order. |
| Outcome: | The proposed model achieves state-of-the-art on the SemEval-2014 word-to-sense similarity task and is available online at http://lcl.uniroma1.it/LSTMEmbed. |
PolyLM: Learning about Polysemy through Language Modeling (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing methods to embed word senses have been overtaken by contextualized embeddings . alan ansell and jim koenig present a method which can be applied to downstream tasks . |
| Approach: | They propose a method which formulates learning sense embeddings as a language modeling problem. |
| Outcome: | The proposed method performs better than existing sense embedding methods on WSI tasks . it matches the current state-of-the-art specialized WSi method despite having six times fewer parameters . |
A Probabilistic Model for Joint Learning of Word Embeddings from Texts and Images (D18-1)
Copied to clipboard
| Challenge: | Existing approaches combine language and perception to infer word embeddings . however, the embeddables produced by such models do not reflect the actual word representations. |
| Approach: | They propose a probabilistic model that integrates linguistic and perceptual inputs to explain observed word-context pairs in a text corpus. |
| Outcome: | The proposed model achieves competitive or stronger results on tasks of assessing pairwise word similarity and image/caption retrieval compared to other state-of-the-art models. |
Retrofitting Word Representations for Unsupervised Sense Aware Word Similarities (L18-1)
Copied to clipboard
| Challenge: | Standard word embeddings lack the ability to distinguish senses of a word by projecting them to exactly one vector. |
| Approach: | They propose to retrofit standard word embeddings to produce sense-aware embeddable vectors using external resources as sense inventories. |
| Outcome: | The proposed method improves word similarity and relatedness scores on multiple word embeddings and established word similarities, sometimes up to an impressive margin of +0.15 Spearman correlation score. |