Word and Document Embedding with vMF-Mixture Priors on Context Word Vectors (P19-1)
Copied to clipboard
| Challenge: | Word embedding models typically learn two types of vectors: target word vectors and context word vector. |
| Approach: | They propose to explicitly impose a cluster structure on context word vectors to improve word embedding models. |
| Outcome: | The proposed model improves word embedding models qualitatively by imposing a cluster structure on the set of context word vectors. |
Similar Papers
Vector of Locally-Aggregated Word Embeddings (VLAWE): A Novel Document-level Representation (N19-1)
Copied to clipboard
| Challenge: | a novel word embedding representation for text documents is proposed . the method is based on the Vector of Locally-Aggregated Descriptors used for image representation . |
| Approach: | They propose a novel representation for text documents based on aggregating word embedding vectors into document embeddables. |
| Outcome: | The proposed representation improves on the Movie Review data set and is 10% better than the state-of-the-art representation. |
Quantifying Context Overlap for Training Word Embeddings (D18-1)
Copied to clipboard
| Challenge: | Experimental results show that word embeddings can be improved using word embeds . word embedings are a popular form of natural language processing . |
| Approach: | They propose to estimate second order co-occurrence relations based on context overlap . they use the augmented data to enhance word embeddings learning . |
| Outcome: | The proposed model improves word vectors for word similarity and downstream NLP tasks. |
Dynamic Contextualized Word Embeddings (2021.acl-long)
Copied to clipboard
| Challenge: | Static word embeddings that represent words by a single vector cannot capture word meaning in different linguistic and extralinguistic contexts. |
| Approach: | They propose dynamic contextualized word embeddings that represent words as a function of linguistic and extralinguistic contexts. |
| Outcome: | The proposed model models time and social space jointly, making them attractive for NLP tasks involving semantic variability. |
Unsupervised Learning of Distributional Relation Vectors (P18-1)
Copied to clipboard
| Challenge: | Existing word embedding models rely on co-occurrence statistics to learn vector representations of word meaning. |
| Approach: | They propose a method which directly learns relation vectors from co-occurrence statistics. |
| Outcome: | The proposed method is based on a variant of GloVe, which has an explicit connection between word vectors and PMI weighted co-occurrence vectors. |
Domain-Specific Word Embeddings with Structure Prediction (2023.tacl-1)
Copied to clipboard
| Challenge: | Current word embedding methods do not provide a way to use or predict information on structure between sub-corpora, time or domain. |
| Approach: | They propose a word embedding method that provides general word representations for the whole corpus, domain-specific representations and embeddable alignment simultaneously. |
| Outcome: | The proposed method provides better performance than baselines on a dataset of science and philosophy articles. |
Better Word Embeddings by Disentangling Contextual n-Gram Information (N19-1)
Copied to clipboard
| Challenge: | Pre-trained word vectors are ubiquitous in Natural Language Processing applications. |
| Approach: | They show that word embeddings with bigram and trigram embedds improve unigram embeds . they claim this removes contextual information from unigrammes, resulting in better unigraph embedders . |
| Outcome: | The proposed model outperforms competing models on a wide variety of tasks. |
Generalizing Word Embeddings using Bag of Subwords (D18-1)
Copied to clipboard
| Challenge: | Existing word embeddings techniques have a fixed vocabulary, i.e., they can only provide vectors over a finite set of common words that appear frequently in a given corpus. |
| Approach: | They propose a subword-level word vector generation model that views words as bags of character n-grams and provides good vectors for rare or unseen words. |
| Outcome: | The proposed model performs state-of-the-art in English word similarity task and in joint prediction of part-of speech tag and morphosyntactic attributes in 23 languages. |
Distilling Relation Embeddings from Pretrained Language Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models capture a surprisingly rich amount of lexical knowledge, but it is unclear to what extent relation embeddings can be used to encode relational knowledge. |
| Approach: | They found that word vector differences capture lexical relations . relationship embeddings can be used to encode relational knowledge . |
| Outcome: | The results are highly competitive on analogy (unsupervised) and relation classification (supervised) benchmarks, even without any task-specific fine-tuning. |
Advances in Pre-Training Distributed Word Representations (L18-1)
Copied to clipboard
| Challenge: | Pre-trained word representations are a building block of many Natural Language Processing and Machine Learning applications. |
| Approach: | They propose to combine known tricks and a set of publicly available pre-trained word vector representations to train high-quality representations. |
| Outcome: | The proposed models outperform the current state of the art on a number of tasks while maintaining a high training speed to scale to massive amount of data. |
Querying Word Embeddings for Similarity and Relatedness (N18-1)
Copied to clipboard
| Challenge: | Word2Vec embeddings have become popular representations of word meaning . similarity between two words is often assumed to be a direction-less measure, whereas relatedness is inherently directional. |
| Approach: | They propose to use word embeddings to predict asymmetric association between words from a dataset of production norms to generate thematically related words. |
| Outcome: | The proposed model predicts asymmetric association between words from a recently published dataset of production norms. |