| Challenge: | Neural word embeddings require lookup and a large memory footprint making it hard to deploy on-device. |
| Approach: | They propose a skip-gram based architecture coupled with Locality-Sensitive Hashing projections to learn efficient dynamically computable representations. |
| Outcome: | The proposed model performs better than previous models on multiple NLP tasks. |
Similar Papers
Disambiguated skip-gram model (D18-1)
Copied to clipboard
| Challenge: | Disambiguated skip-gram is a neural-probabilistic model for learning multi-sense word embeddings. |
| Approach: | They propose a model that is end-to-end differentiable and can be interpreted as a feed-forward neural network. |
| Outcome: | The proposed model improves state-of-the-art in word sense induction benchmarks. |
Advances in Pre-Training Distributed Word Representations (L18-1)
Copied to clipboard
| Challenge: | Pre-trained word representations are a building block of many Natural Language Processing and Machine Learning applications. |
| Approach: | They propose to combine known tricks and a set of publicly available pre-trained word vector representations to train high-quality representations. |
| Outcome: | The proposed models outperform the current state of the art on a number of tasks while maintaining a high training speed to scale to massive amount of data. |
LSTMEmbed: Learning Word and Sense Representations from a Large Semantically Annotated Corpus with Long Short-Term Memories (P19-1)
Copied to clipboard
| Challenge: | Recent work has focused on vector representations which capture different meanings, i.e., senses, of words. |
| Approach: | They propose a bidirectional LSTM model which learns word senses from semantically annotated corpora by focusing on word order. |
| Outcome: | The proposed model achieves state-of-the-art on the SemEval-2014 word-to-sense similarity task and is available online at http://lcl.uniroma1.it/LSTMEmbed. |
Urdu Word Embeddings (L18-1)
Copied to clipboard
| Challenge: | Recent advances in distributional semantics have led to the rise of neural network-based models that use unsupervised learning to represent words as dense, distributed vectors, called 'word embeddings' embedders hold key to improving natural language processing for low-resource languages, since they require significant time and manpower. |
| Approach: | They train a skip-gram model on 140 million Urdu words to create the first large-scale word embeddings for the Urdu language. |
| Outcome: | The proposed models capture high degree of syntactic and semantic similarity between words and are able to generalize well on the Urdu translation task. |
Cross-Lingual Syntactic Transfer through Unsupervised Adaptation of Invertible Projections (P19-1)
Copied to clipboard
| Challenge: | Current systems for syntactic analysis tasks rely heavily on large scale annotated data. |
| Approach: | They propose to learn a generative model with a structured prior that uses labeled source and unlabeled target data jointly. |
| Outcome: | The proposed model improves on part-of-speech tagging and dependency parsing tasks on English as the only source corpus and on a wide range of target languages. |
Memory, Show the Way: Memory Based Few Shot Word Representation Learning (D18-1)
Copied to clipboard
| Challenge: | Existing word embedding methods for distributed semantic models require limited examples to learn a high quality representation. |
| Approach: | They propose a memory-based embedding learning method capable of acquiring word representations from limited context. |
| Outcome: | The proposed method delivers impressive performance on two challenging few-shot word similarity tasks. |
Language Representation Projection: Can We Transfer Factual Knowledge across Languages in Multilingual Language Models? (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies show that multilingual pretrained models can recall factual knowledge without additional fine-tuning. |
| Approach: | They propose two parameter-free language representation projection modules to transfer factual knowledge between English and non-English languages. |
| Outcome: | The proposed modules improve factual knowledge retrieval accuracy and transferability across diverse non-English languages. |
A La Carte Embedding: Cheap but Effective Induction of Semantic Feature Vectors (P18-1)
Copied to clipboard
| Challenge: | Existing word2vec-based methods for learning rare or unseen words have been criticized for degrading performance in small corpus settings. |
| Approach: | They propose a la carte embedding method that relies on a linear transformation that is efficiently learnable using pretrained word vectors and linear regression. |
| Outcome: | The proposed method is based on a new dataset showing that it can be used when a word is encountered even if only a single usage example is available. |
Segmentation-free compositional n-gram embedding (N19-1)
Copied to clipboard
| Challenge: | Existing word embedding models depend on word segmentation, but this method is difficult when corpora written in noisy or unsegmented languages. |
| Approach: | They propose a new method that models words, phrases and sentences seamlessly without word segmentation. |
| Outcome: | The proposed method is very effective for noisy corpora written in unsegmented languages such as Chinese and Japanese. |
Semantic Aware Linear Transfer by Recycling Pre-trained Language Models for Cross-lingual Transfer (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly incorporating multilingual capabilities, fueling the demand to transfer them into target language-specific models. |
| Approach: | They propose a novel cross-lingual transfer technique that recycles embeddings from target language Pre-trained Language Models to transmit deep representational strengths to LLMs. |
| Outcome: | The proposed technique outperforms existing methods in cross-lingual understanding setups and achieves faster convergence and lower loss during language adaptation. |