Improving Word Embeddings through Iterative Refinement of Word- and Character-level Models (2020.coling-main)
Copied to clipboard
| Challenge: | Embedding of rare and out-of-vocabulary words is an important open NLP problem . standard embedding models are not useful for recommending jobs to users with rare or unseen words . |
| Approach: | They propose to train a character-level neural network to reproduce word embeddings . they then use the model to assign vectors to any input string, including rare words . |
| Outcome: | The proposed method outperforms existing methods on word similarity data sets and can be applied to job title normalization in the e-recruitment domain. |
Similar Papers
Robust Backed-off Estimation of Out-of-Vocabulary Embeddings (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to solving out-of-vocabulary (OOV) words use subwords to represent oov words with a bag of subword. |
| Approach: | They propose a method to estimate oov word embeddings by referring to pre-trained word embeds for known words with similar surfaces to target ov words. |
| Outcome: | The proposed method improves word similarity tasks and biomedical tasks even with weak baselines. |
Representation Learning for Unseen Words by Bridging Subwords to Semantic Networks (2020.lrec-1)
Copied to clipboard
| Challenge: | Pre-trained word embeddings only include words that appeared in corpora where pre-tried embedds are learned. |
| Approach: | They propose a method to represent out-of-vocabulary words using subword information and knowledge. |
| Outcome: | The proposed method improves performance over baselines that only use subwords or knowledge to represent OOV words. |
Generalizing Word Embeddings using Bag of Subwords (D18-1)
Copied to clipboard
| Challenge: | Existing word embeddings techniques have a fixed vocabulary, i.e., they can only provide vectors over a finite set of common words that appear frequently in a given corpus. |
| Approach: | They propose a subword-level word vector generation model that views words as bags of character n-grams and provides good vectors for rare or unseen words. |
| Outcome: | The proposed model performs state-of-the-art in English word similarity task and in joint prediction of part-of speech tag and morphosyntactic attributes in 23 languages. |
Handling Out-Of-Vocabulary Problem in Hangeul Word Embeddings (2021.eacl-main)
Copied to clipboard
| Challenge: | Word embedding is considered an essential factor in improving the performance of various Natural Language Processing (NLP) models. |
| Approach: | They propose a Hangeul word embedding model that infers original word embeds from typos while maintaining high performance. |
| Outcome: | The proposed model performs well against typos while maintaining high performance. |
Imputing Out-of-Vocabulary Embeddings with LOVE Makes LanguageModels Robust with Little Cost (2022.acl-long)
Copied to clipboard
| Challenge: | State-of-the-art NLP systems are brittle when faced with Out-ofVocabulary words . we present a framework that extends word embeddings and makes them robust to OOV . |
| Approach: | They propose a framework that extends existing word embeddings and makes them robust to OOV. |
| Outcome: | The proposed model performs better on original datasets and corrupted variants than previous competitors. |
Few-Shot Representation Learning for Out-Of-Vocabulary Words (P19-1)
Copied to clipboard
| Challenge: | Existing methods for learning word embedding assume there are enough occurrences for each word in the corpus to accurately estimate the representation of words. |
| Approach: | They propose to fit a representation function to predict an oracle embedding vector based on limited contexts. |
| Outcome: | The proposed model outperforms existing methods in constructing an accurate embedding for OOV words and improves downstream tasks when the embeddable is utilized. |
Subword-based Compact Reconstruction of Word Embeddings (N19-1)
Copied to clipboard
| Challenge: | Existing word-based word embeddings are based on subword information and memory-shared embeddables. |
| Approach: | They propose a method for reconstructing pre-trained word embeddings using subword information using memory-shared embedds and a variant of the key-value-query self-attention mechanism. |
| Outcome: | The proposed method can imitate well-trained word embeddings in a small fixed space while preventing quality degradation across several linguistic benchmark datasets. |
Refining Pretrained Word Embeddings Using Layer-wise Relevance Propagation (D18-1)
Copied to clipboard
| Challenge: | Recent research trend is to refine or fine-tune pretrained word embeddings. |
| Approach: | They propose a method for refining pretrained word embeddings using layer-wise relevance propagation using a neural network. |
| Outcome: | The proposed method achieves higher performance than the original vectors. |
Embeddings in Natural Language Processing (2020.coling-tutorials)
Copied to clipboard
| Challenge: | Embeddings have been a key topic of interest in NLP for the past decade . a quick warm-up introduction to NLP and why it is important to have a semantic comprehension of texts . |
| Approach: | This tutorial will provide a high-level synthesis of the main embedding techniques in NLP . it will start with word embedds and then move to other types of embeddable vectors . |
| Outcome: | This tutorial will provide a high-level synthesis of the main embedding techniques in NLP . it will start with word embedds and move to other types of embeddable representations . |
Better Word Embeddings by Disentangling Contextual n-Gram Information (N19-1)
Copied to clipboard
| Challenge: | Pre-trained word vectors are ubiquitous in Natural Language Processing applications. |
| Approach: | They show that word embeddings with bigram and trigram embedds improve unigram embeds . they claim this removes contextual information from unigrammes, resulting in better unigraph embedders . |
| Outcome: | The proposed model outperforms competing models on a wide variety of tasks. |