| Challenge: | Existing word embedding models rely on co-occurrence statistics to learn vector representations of word meaning. |
| Approach: | They propose a method which directly learns relation vectors from co-occurrence statistics. |
| Outcome: | The proposed method is based on a variant of GloVe, which has an explicit connection between word vectors and PMI weighted co-occurrence vectors. |
Similar Papers
Relational Word Embeddings (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to learn word embeddings rely on external knowledge bases . however, they are limited by the amount of available relational knowledge . |
| Approach: | They propose to encode relational knowledge in a separate word embedding . this is complementary to a standard word embedded from co-occurrence statistics . |
| Outcome: | The proposed word embedding is complementary to a standard word embed. |
Word and Document Embedding with vMF-Mixture Priors on Context Word Vectors (P19-1)
Copied to clipboard
| Challenge: | Word embedding models typically learn two types of vectors: target word vectors and context word vector. |
| Approach: | They propose to explicitly impose a cluster structure on context word vectors to improve word embedding models. |
| Outcome: | The proposed model improves word embedding models qualitatively by imposing a cluster structure on the set of context word vectors. |
Embedding Words in Non-Vector Space with Unsupervised Graph Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | GraphGlove is an unsupervised graph word representations that are learned end-to-end. |
| Approach: | They propose a method to learn weighted graph word representations end-to-end using a weighteable weighte . they adopt a hierarchical graph representation method and modify the GloVe training algorithm to learn graph representations. |
| Outcome: | The proposed method outperforms vector-based methods on word similarity and analogy tasks. |
Relation Induction in Word Embeddings Revisited (C18-1)
Copied to clipboard
| Challenge: | Existing approaches to relation induction are based on vector translations, but they are often inadequate for knowledge base completion. |
| Approach: | They propose to use Gaussian to explicitly model the variability of translations and Bayesian linear regression to encode the assumption that there is a linear relationship between the vector representations of related words. |
| Outcome: | The proposed models are based on translations but use Gaussian to model the variability of translations and encode soft constraints on the source and target words that may be chosen. |
SeVeN: Augmenting Word Embeddings with Unsupervised Relation Vectors (C18-1)
Copied to clipboard
| Challenge: | Word embeddings use fixed-dimensional vectors to represent the meaning of words. |
| Approach: | They propose a pipeline for learning relation vectors based on word vector averaging and an ad hoc autoencoder. |
| Outcome: | The proposed pipeline can capture aspects of word meaning complementary to word embeddings. |
Distilling Relation Embeddings from Pretrained Language Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models capture a surprisingly rich amount of lexical knowledge, but it is unclear to what extent relation embeddings can be used to encode relational knowledge. |
| Approach: | They found that word vector differences capture lexical relations . relationship embeddings can be used to encode relational knowledge . |
| Outcome: | The results are highly competitive on analogy (unsupervised) and relation classification (supervised) benchmarks, even without any task-specific fine-tuning. |
Advances in Pre-Training Distributed Word Representations (L18-1)
Copied to clipboard
| Challenge: | Pre-trained word representations are a building block of many Natural Language Processing and Machine Learning applications. |
| Approach: | They propose to combine known tricks and a set of publicly available pre-trained word vector representations to train high-quality representations. |
| Outcome: | The proposed models outperform the current state of the art on a number of tasks while maintaining a high training speed to scale to massive amount of data. |
Quantifying Context Overlap for Training Word Embeddings (D18-1)
Copied to clipboard
| Challenge: | Experimental results show that word embeddings can be improved using word embeds . word embedings are a popular form of natural language processing . |
| Approach: | They propose to estimate second order co-occurrence relations based on context overlap . they use the augmented data to enhance word embeddings learning . |
| Outcome: | The proposed model improves word vectors for word similarity and downstream NLP tasks. |
Global Relation Embedding for Relation Extraction (N18-1)
Copied to clipboard
| Challenge: | Existing methods to extract textual relations with distant supervision are limited by their reliance on supervised training data. |
| Approach: | They propose to embed relations with global statistics of relations to combat the wrong labeling problem of distant supervision. |
| Outcome: | The proposed method is more robust to training noise introduced by distant supervision and improves relation extraction models. |
Better Word Embeddings by Disentangling Contextual n-Gram Information (N19-1)
Copied to clipboard
| Challenge: | Pre-trained word vectors are ubiquitous in Natural Language Processing applications. |
| Approach: | They show that word embeddings with bigram and trigram embedds improve unigram embeds . they claim this removes contextual information from unigrammes, resulting in better unigraph embedders . |
| Outcome: | The proposed model outperforms competing models on a wide variety of tasks. |