HG2Vec: Improved Word Embeddings from Dictionary and Thesaurus Based Heterogeneous Graph (2022.coling-1)
Copied to clipboard
| Challenge: | Existing models that learn word embeddings rely on a large corpus of data . however, these models require massive time and space for data pre-processing and training . |
| Approach: | They propose a model that learns word embeddings utilizing only dictionaries and thesauri . they exploit a new context-focused loss model that models transitive relationships between word pairs . |
| Outcome: | The proposed model reaches the state-of-art on multiple word similarity and relatedness benchmarks. |
Similar Papers
Wikipedia2Vec: An Efficient Toolkit for Learning and Visualizing the Embeddings of Words and Entities from Wikipedia (2020.emnlp-demos)
Copied to clipboard
Ikuya Yamada, Akari Asai, Jin Sakuma, Hiroyuki Shindo, Hideaki Takeda, Yoshiyasu Takefuji, Yuji Matsumoto
| Challenge: | Existing tools for learning the embeddings of words and entities from Wikipedia are not yet available. |
| Approach: | They propose a Python-based tool for learning Wikipedia embeddings from Wikipedia . they use a Wikipedia dump file as an argument to issue a single command . |
| Outcome: | The proposed tool achieves state-of-the-art results on the KORE entity relatedness dataset and competitive results on benchmark datasets. |
Using Context-to-Vector with Graph Retrofitting to Improve Word Embeddings (2022.acl-long)
Copied to clipboard
| Challenge: | Contextualized embeddings are expensive and resource-demanding, hence environmentally unfriendly. |
| Approach: | They propose a method to convert contextualized embeddings from pre-trained models into static embeddables using synonym knowledge and weighted vector distribution. |
| Outcome: | The proposed method outperforms baseline embeddings by a large margin through extrinsic and intrinsic tasks. |
Building Static Embeddings from Contextual Ones: Is It Useful for Building Distributional Thesauri? (2022.lrec-1)
Copied to clipboard
| Challenge: | contextual language models are dominant in the field of Natural Language Processing, but they are not suitable for all uses. |
| Approach: | They propose a method for building word or type-level embeddings from contextual models . they evaluate a large set of English nouns from the perspective of extracting semantic similarity relations . |
| Outcome: | The proposed method can be used to build word or type embeddings from contextual models . it can be exploited for a wide set of English nouns, showing it can improve distributional thesauri . |
Embedding Words in Non-Vector Space with Unsupervised Graph Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | GraphGlove is an unsupervised graph word representations that are learned end-to-end. |
| Approach: | They propose a method to learn weighted graph word representations end-to-end using a weighteable weighte . they adopt a hierarchical graph representation method and modify the GloVe training algorithm to learn graph representations. |
| Outcome: | The proposed method outperforms vector-based methods on word similarity and analogy tasks. |
Embeddings in Natural Language Processing (2020.coling-tutorials)
Copied to clipboard
| Challenge: | Embeddings have been a key topic of interest in NLP for the past decade . a quick warm-up introduction to NLP and why it is important to have a semantic comprehension of texts . |
| Approach: | This tutorial will provide a high-level synthesis of the main embedding techniques in NLP . it will start with word embedds and then move to other types of embeddable vectors . |
| Outcome: | This tutorial will provide a high-level synthesis of the main embedding techniques in NLP . it will start with word embedds and move to other types of embeddable representations . |
Auto-Encoding Dictionary Definitions into Consistent Word Embeddings (D18-1)
Copied to clipboard
| Challenge: | Monolingual dictionaries are widespread and semantically rich resources. |
| Approach: | They propose a model that learns to compute word embeddings by processing dictionary definitions and trying to reconstruct them. |
| Outcome: | The proposed model shows strong performance when trained exclusively on dictionary data and generalizes in one shot. |
Better Word Embeddings by Disentangling Contextual n-Gram Information (N19-1)
Copied to clipboard
| Challenge: | Pre-trained word vectors are ubiquitous in Natural Language Processing applications. |
| Approach: | They show that word embeddings with bigram and trigram embedds improve unigram embeds . they claim this removes contextual information from unigrammes, resulting in better unigraph embedders . |
| Outcome: | The proposed model outperforms competing models on a wide variety of tasks. |
Word Mover’s Embedding: From Word2Vec to Document Embedding (D18-1)
Copied to clipboard
Lingfei Wu, Ian En-Hsu Yen, Kun Xu, Fangli Xu, Avinash Balakrishnan, Pin-Yu Chen, Pradeep Ravikumar, Michael J. Witbrock
| Challenge: | Recent work has demonstrated that Word Mover’s Distance (WMD) that aligns semantically similar words yields unprecedented KNN classification accuracy. |
| Approach: | They propose a Word Mover’s Distance (WMD) method that aligns semantically similar words to generate unsupervised sentences or documents embeddings. |
| Outcome: | The proposed method consistently outperforms state-of-the-art techniques on 9 benchmark text classification datasets and 22 textual similarity tasks. |
Domain-Specific Word Embeddings with Structure Prediction (2023.tacl-1)
Copied to clipboard
| Challenge: | Current word embedding methods do not provide a way to use or predict information on structure between sub-corpora, time or domain. |
| Approach: | They propose a word embedding method that provides general word representations for the whole corpus, domain-specific representations and embeddable alignment simultaneously. |
| Outcome: | The proposed method provides better performance than baselines on a dataset of science and philosophy articles. |
Beyond Offline Mapping: Learning Cross-lingual Word Embeddings through Context Anchoring (2021.acl-long)
Copied to clipboard
| Challenge: | Recent research on cross-lingual word embeddings has been dominated by unsupervised mapping approaches that align monolingual embedders. |
| Approach: | They propose an unsupervised mapping approach that fixes fixed embeddings and learns new ones for the source language that are aligned with them. |
| Outcome: | The proposed method outperforms conventional mapping methods on bilingual lexicon induction and obtains competitive results in the downstream XNLI task. |