| Challenge: | Experimental results show that word embeddings can be improved using word embeds . word embedings are a popular form of natural language processing . |
| Approach: | They propose to estimate second order co-occurrence relations based on context overlap . they use the augmented data to enhance word embeddings learning . |
| Outcome: | The proposed model improves word vectors for word similarity and downstream NLP tasks. |
Similar Papers
Querying Word Embeddings for Similarity and Relatedness (N18-1)
Copied to clipboard
| Challenge: | Word2Vec embeddings have become popular representations of word meaning . similarity between two words is often assumed to be a direction-less measure, whereas relatedness is inherently directional. |
| Approach: | They propose to use word embeddings to predict asymmetric association between words from a dataset of production norms to generate thematically related words. |
| Outcome: | The proposed model predicts asymmetric association between words from a recently published dataset of production norms. |
On the Correlation of Word Embedding Evaluation Metrics (2020.lrec-1)
Copied to clipboard
| Challenge: | Word embeddings are geometrical representations of word paradigmatics and syntagmatics. |
| Approach: | They propose to investigate evaluation metrics on various datasets to find correlations . they propose a fast solution to select the best word embeddings among many others . |
| Outcome: | The proposed method could be used to select the best word embeddings among many others. |
Word and Document Embedding with vMF-Mixture Priors on Context Word Vectors (P19-1)
Copied to clipboard
| Challenge: | Word embedding models typically learn two types of vectors: target word vectors and context word vector. |
| Approach: | They propose to explicitly impose a cluster structure on context word vectors to improve word embedding models. |
| Outcome: | The proposed model improves word embedding models qualitatively by imposing a cluster structure on the set of context word vectors. |
Joint Embedding of Words and Labels for Text Classification (P18-1)
Copied to clipboard
Guoyin Wang, Chunyuan Li, Wenlin Wang, Yizhe Zhang, Dinghan Shen, Xinyuan Zhang, Ricardo Henao, Lawrence Carin
| Challenge: | Existing approaches to text classification use word embeddings to capture semantic regularities between words. |
| Approach: | They propose to view text classification as a label-word joint embedding problem . they use a framework that measures compatibility between text sequences and labels . |
| Outcome: | The proposed framework outperforms the state-of-the-art methods on large text datasets. |
A Deeper Look into Dependency-Based Word Embeddings (N18-4)
Copied to clipboard
| Challenge: | Word embeddings trained with dependency contexts excel at different tasks, and enhanced dependencies often improve performance. |
| Approach: | They propose to use dependency-based word embeddings to capture semantic similarity rather than relatedness. |
| Outcome: | The results show that word embeddings trained with Universal and Stanford dependencies excel at different tasks and that enhanced dependencies often improve performance. |
A Rank-Based Similarity Metric for Word Embeddings (P18-2)
Copied to clipboard
| Challenge: | Word Embeddings have become a standard for word representations, with vector cosine being the only similarity metric. |
| Approach: | They propose to use rank-based similarity estimation metrics to measure word similarity . they find WE outperforms vector cosine in the recent outlier detection task . |
| Outcome: | The proposed rank-based measure outperforms vector cosine in the recent outlier detection task. |
Relation Induction in Word Embeddings Revisited (C18-1)
Copied to clipboard
| Challenge: | Existing approaches to relation induction are based on vector translations, but they are often inadequate for knowledge base completion. |
| Approach: | They propose to use Gaussian to explicitly model the variability of translations and Bayesian linear regression to encode the assumption that there is a linear relationship between the vector representations of related words. |
| Outcome: | The proposed models are based on translations but use Gaussian to model the variability of translations and encode soft constraints on the source and target words that may be chosen. |
Entity or Relation Embeddings? An Analysis of Encoding Strategies for Relation Extraction (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to relation extraction use concatenating embeddings of head and tail entities . however, such representations capture the types of the entities involved, leading to false positives and confusion between relations involving entities of the same type. |
| Approach: | They propose a model which combines [MASK] embeddings with entity embedds to learn relation embeddations. |
| Outcome: | The proposed model outperforms the state-of-the-art on several benchmarks . it uses a self-supervised pre-training strategy which further improves the results. |
Towards Qualitative Word Embeddings Evaluation: Measuring Neighbors Variation (N18-4)
Copied to clipboard
| Challenge: | Using extrinsic evaluation methods, embeddings are evaluated on a specific task such as part-of-speech tagging or named-entity recognition. |
| Approach: | They propose a method to study the variation between word embeddings models trained with only one parameter by observing the distributional neighbors variation. |
| Outcome: | The proposed method shows that changing only one parameter can have a massive impact on a given semantic space. |
Distilling Relation Embeddings from Pretrained Language Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models capture a surprisingly rich amount of lexical knowledge, but it is unclear to what extent relation embeddings can be used to encode relational knowledge. |
| Approach: | They found that word vector differences capture lexical relations . relationship embeddings can be used to encode relational knowledge . |
| Outcome: | The results are highly competitive on analogy (unsupervised) and relation classification (supervised) benchmarks, even without any task-specific fine-tuning. |