| Challenge: | Word2Vec embeddings have become popular representations of word meaning . similarity between two words is often assumed to be a direction-less measure, whereas relatedness is inherently directional. |
| Approach: | They propose to use word embeddings to predict asymmetric association between words from a dataset of production norms to generate thematically related words. |
| Outcome: | The proposed model predicts asymmetric association between words from a recently published dataset of production norms. |
Similar Papers
Quantifying Context Overlap for Training Word Embeddings (D18-1)
Copied to clipboard
| Challenge: | Experimental results show that word embeddings can be improved using word embeds . word embedings are a popular form of natural language processing . |
| Approach: | They propose to estimate second order co-occurrence relations based on context overlap . they use the augmented data to enhance word embeddings learning . |
| Outcome: | The proposed model improves word vectors for word similarity and downstream NLP tasks. |
Understanding Undesirable Word Embedding Associations (P19-1)
Copied to clipboard
| Challenge: | Word embeddings are often criticized for capturing undesirable word associations such as gender stereotypes. |
| Approach: | They propose to use subspace projection to debias vectors post hoc using a model that implicitly does matrix factorization to debunk gender bias. |
| Outcome: | The proposed test overestimates gender bias in word embeddings by using subspace projection, a method that is widely used in training. |
Analyzing the Surprising Variability in Word Embedding Stability Across Languages (2021.emnlp-main)
Copied to clipboard
| Challenge: | Word embeddings are powerful representations that form the foundation of many natural language processing architectures. |
| Approach: | They explore word embedding stability in a wide range of languages to gain insight into their stability. |
| Outcome: | The proposed results provide insights into word embedding stability in English and other languages. |
Lexical Relation Mining in Neural Word Embeddings (2020.coling-main)
Copied to clipboard
| Challenge: | Conventionally, lexical relations in word vector space have been defined by collections of relatively consistent relationships, or vector offsets, between word-pairs. |
| Approach: | They propose to use Word2Vec space of word-pairs to find lexical relations . they also demonstrate a method for approximating the presence of syntactic and semantic relations based on word vectors extracted from word embeddings. |
| Outcome: | The proposed method outperforms other validated methods in the presence of noisy offsets. |
How to represent a word and predict it, too: Improving tied architectures for language modelling (D18-1)
Copied to clipboard
| Challenge: | Recent state-of-the-art models use word embeddings as input and output mappings instead of tied models. |
| Approach: | They propose to decouple hidden state from word embedding prediction . they extend their proposed modification to word2vec models . |
| Outcome: | The proposed architectures achieve comparable or better results compared to previous models without tying . the proposed architecture reduces parameters, enabling more compact models and faster learning. |
Do Word Embeddings Capture Spelling Variation? (2020.coling-main)
Copied to clipboard
| Challenge: | Using word embeddings, we analyze spelling variation in word embeds trained on Twitter and Reddit data. |
| Approach: | They propose a new perspective on the analysis of word embeddings by focusing on spelling variation. |
| Outcome: | The proposed analysis shows that word embeddings encode spelling variation patterns of various types to some extent, even when trained using the skipgram model. |
Embeddings in Natural Language Processing (2020.coling-tutorials)
Copied to clipboard
| Challenge: | Embeddings have been a key topic of interest in NLP for the past decade . a quick warm-up introduction to NLP and why it is important to have a semantic comprehension of texts . |
| Approach: | This tutorial will provide a high-level synthesis of the main embedding techniques in NLP . it will start with word embedds and then move to other types of embeddable vectors . |
| Outcome: | This tutorial will provide a high-level synthesis of the main embedding techniques in NLP . it will start with word embedds and move to other types of embeddable representations . |
On the Correlation of Word Embedding Evaluation Metrics (2020.lrec-1)
Copied to clipboard
| Challenge: | Word embeddings are geometrical representations of word paradigmatics and syntagmatics. |
| Approach: | They propose to investigate evaluation metrics on various datasets to find correlations . they propose a fast solution to select the best word embeddings among many others . |
| Outcome: | The proposed method could be used to select the best word embeddings among many others. |
Relation Induction in Word Embeddings Revisited (C18-1)
Copied to clipboard
| Challenge: | Existing approaches to relation induction are based on vector translations, but they are often inadequate for knowledge base completion. |
| Approach: | They propose to use Gaussian to explicitly model the variability of translations and Bayesian linear regression to encode the assumption that there is a linear relationship between the vector representations of related words. |
| Outcome: | The proposed models are based on translations but use Gaussian to model the variability of translations and encode soft constraints on the source and target words that may be chosen. |
Using Context-to-Vector with Graph Retrofitting to Improve Word Embeddings (2022.acl-long)
Copied to clipboard
| Challenge: | Contextualized embeddings are expensive and resource-demanding, hence environmentally unfriendly. |
| Approach: | They propose a method to convert contextualized embeddings from pre-trained models into static embeddables using synonym knowledge and weighted vector distribution. |
| Outcome: | The proposed method outperforms baseline embeddings by a large margin through extrinsic and intrinsic tasks. |