| Challenge: | Magnitude is an open source Python package that performs common operations up to 6,000 times faster than Gensim. |
| Approach: | They present a Python tool for utilizing vector embeddings that performs common operations up to 6,000 times faster than Gensim. |
| Outcome: | The Magnitude package performs common operations up to 6,000 times faster than Gensim and introduces several novel features for improved robustness like out-of-vocabulary lookups. |
Similar Papers
Embeddings in Natural Language Processing (2020.coling-tutorials)
Copied to clipboard
| Challenge: | Embeddings have been a key topic of interest in NLP for the past decade . a quick warm-up introduction to NLP and why it is important to have a semantic comprehension of texts . |
| Approach: | This tutorial will provide a high-level synthesis of the main embedding techniques in NLP . it will start with word embedds and then move to other types of embeddable vectors . |
| Outcome: | This tutorial will provide a high-level synthesis of the main embedding techniques in NLP . it will start with word embedds and move to other types of embeddable representations . |
Deconstructing word embedding algorithms (2020.emnlp-main)
Copied to clipboard
| Challenge: | Word embeddings are reliable feature representations of words used in many NLP tasks today. |
| Approach: | They propose to deconstruct Word2vec, GloVe and others into a common form . they propose to generalize several word embedding algorithms into . a low rank embedder framework is proposed to generalise the algorithms into one common form. |
| Outcome: | The proposed framework can be used to make word embeddings more performant. |
COIN – an Inexpensive and Strong Baseline for Predicting Out of Vocabulary Word Embeddings (2022.coling-1)
Copied to clipboard
| Challenge: | Word embedding models only include terms that occur a sufficient number of times in training corpora. |
| Approach: | They propose a method for predicting word embeddings for out of vocabulary terms using word2vec. |
| Outcome: | The proposed method surpasses several methods on benchmark tasks and is inexpensive to compute. |
Wikipedia2Vec: An Efficient Toolkit for Learning and Visualizing the Embeddings of Words and Entities from Wikipedia (2020.emnlp-demos)
Copied to clipboard
Ikuya Yamada, Akari Asai, Jin Sakuma, Hiroyuki Shindo, Hideaki Takeda, Yoshiyasu Takefuji, Yuji Matsumoto
| Challenge: | Existing tools for learning the embeddings of words and entities from Wikipedia are not yet available. |
| Approach: | They propose a Python-based tool for learning Wikipedia embeddings from Wikipedia . they use a Wikipedia dump file as an argument to issue a single command . |
| Outcome: | The proposed tool achieves state-of-the-art results on the KORE entity relatedness dataset and competitive results on benchmark datasets. |
Block-wise Word Embedding Compression Revisited: Better Weighting and Structuring (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for word embedding compression are limited . word embeds have a considerable size and need to be compressed to deploy on edge devices . |
| Approach: | They propose a block-wise low-rank approximation method for word embedding called GroupReduce . they propose 'frequency-inverse document frequency method' and a differentiable method for weighting . |
| Outcome: | The proposed algorithm more effectively finds word weights than competitors in most cases. |
Domain-Specific Word Embeddings with Structure Prediction (2023.tacl-1)
Copied to clipboard
| Challenge: | Current word embedding methods do not provide a way to use or predict information on structure between sub-corpora, time or domain. |
| Approach: | They propose a word embedding method that provides general word representations for the whole corpus, domain-specific representations and embeddable alignment simultaneously. |
| Outcome: | The proposed method provides better performance than baselines on a dataset of science and philosophy articles. |
Do We Really Need All Those Dimensions? An Intrinsic Evaluation Framework for Compressed Embeddings (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing evaluation methods for compressed text embeddings are either expensive or too simplistic. |
| Approach: | They propose a task-agnostic intrinsic evaluation framework that provides a reliable proxy for downstream performance. |
| Outcome: | The proposed framework provides a reliable proxy for downstream performance. |
Word Mover’s Embedding: From Word2Vec to Document Embedding (D18-1)
Copied to clipboard
Lingfei Wu, Ian En-Hsu Yen, Kun Xu, Fangli Xu, Avinash Balakrishnan, Pin-Yu Chen, Pradeep Ravikumar, Michael J. Witbrock
| Challenge: | Recent work has demonstrated that Word Mover’s Distance (WMD) that aligns semantically similar words yields unprecedented KNN classification accuracy. |
| Approach: | They propose a Word Mover’s Distance (WMD) method that aligns semantically similar words to generate unsupervised sentences or documents embeddings. |
| Outcome: | The proposed method consistently outperforms state-of-the-art techniques on 9 benchmark text classification datasets and 22 textual similarity tasks. |
Generalizing Word Embeddings using Bag of Subwords (D18-1)
Copied to clipboard
| Challenge: | Existing word embeddings techniques have a fixed vocabulary, i.e., they can only provide vectors over a finite set of common words that appear frequently in a given corpus. |
| Approach: | They propose a subword-level word vector generation model that views words as bags of character n-grams and provides good vectors for rare or unseen words. |
| Outcome: | The proposed model performs state-of-the-art in English word similarity task and in joint prediction of part-of speech tag and morphosyntactic attributes in 23 languages. |
FLAIR: An Easy-to-Use Framework for State-of-the-Art NLP (N19-4)
Copied to clipboard
| Challenge: | Existing approaches combine word embeddings with character-level features to model additional features such as subword structures and meaning ambiguity. |
| Approach: | They present FLAIR, an NLP framework that enables embeddings of word and document data . they propose a hierarchical learning architecture that concatenates output states of a character-level CNN or RNN with the output states from a task data. |
| Outcome: | The proposed framework hides embedding-specific engineering complexity and allows researchers to "mix and match" various embeddables with little effort. |