Glyph2Vec: Learning Chinese Out-of-Vocabulary Word Embedding from Glyphs (2020.acl-main)
Copied to clipboard
| Challenge: | Chinese NLP applications that rely on large text often contain huge amounts of vocabulary which are sparse in corpus. |
| Approach: | They propose a multi-modal model that extracts visual features from Chinese word glyphs to expand current word embedding space without accessing any corpus. |
| Outcome: | The proposed model can embed words in Chinese without accessing corpus without a corpus. |
Similar Papers
COIN – an Inexpensive and Strong Baseline for Predicting Out of Vocabulary Word Embeddings (2022.coling-1)
Copied to clipboard
| Challenge: | Word embedding models only include terms that occur a sufficient number of times in training corpora. |
| Approach: | They propose a method for predicting word embeddings for out of vocabulary terms using word2vec. |
| Outcome: | The proposed method surpasses several methods on benchmark tasks and is inexpensive to compute. |
Embeddings in Natural Language Processing (2020.coling-tutorials)
Copied to clipboard
| Challenge: | Embeddings have been a key topic of interest in NLP for the past decade . a quick warm-up introduction to NLP and why it is important to have a semantic comprehension of texts . |
| Approach: | This tutorial will provide a high-level synthesis of the main embedding techniques in NLP . it will start with word embedds and then move to other types of embeddable vectors . |
| Outcome: | This tutorial will provide a high-level synthesis of the main embedding techniques in NLP . it will start with word embedds and move to other types of embeddable representations . |
Segmentation-free compositional n-gram embedding (N19-1)
Copied to clipboard
| Challenge: | Existing word embedding models depend on word segmentation, but this method is difficult when corpora written in noisy or unsegmented languages. |
| Approach: | They propose a new method that models words, phrases and sentences seamlessly without word segmentation. |
| Outcome: | The proposed method is very effective for noisy corpora written in unsegmented languages such as Chinese and Japanese. |
CWSeg: An Efficient and General Approach to Chinese Word Segmentation (2023.acl-industry)
Copied to clipboard
| Challenge: | Existing methods for Chinese word segmentation have achieved state-of-the-art performance, but they pose challenges in the deployment. |
| Approach: | They propose to augment PLM-based Chinese word segmentation schemes by developing cohort training and versatile decoding strategies. |
| Outcome: | The proposed model can be used to augment existing PLM-based models and improve their performance on Chinese LLaMA and Alpaca datasets. |
Better Word Embeddings by Disentangling Contextual n-Gram Information (N19-1)
Copied to clipboard
| Challenge: | Pre-trained word vectors are ubiquitous in Natural Language Processing applications. |
| Approach: | They show that word embeddings with bigram and trigram embedds improve unigram embeds . they claim this removes contextual information from unigrammes, resulting in better unigraph embedders . |
| Outcome: | The proposed model outperforms competing models on a wide variety of tasks. |
Unsupervised Cross-Lingual Representation Learning (P19-4)
Copied to clipboard
| Challenge: | a comprehensive survey of cutting-edge weakly-supervised and unsupervised cross-lingual word representations is presented . |
| Approach: | This tutorial provides a comprehensive survey of recent work on weakly-supervised and unsupervised cross-lingual word representations. |
| Outcome: | This tutorial provides a comprehensive survey of cutting-edge weakly-supervised and unsupervised word representations. |
Improving Low Compute Language Modeling with In-Domain Embedding Initialisation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to train language models on in-domain data are limited. |
| Approach: | They propose to initialise and freeze in-domain embeddings to provide a useful representation of rare words in English . they find that the standard configuration is not optimal when rare words are present . |
| Outcome: | The proposed approach improves language modeling by providing a useful representation of rare words in English. |
KIT-Multi: A Translation-Oriented Multilingual Embedding Corpus (L18-1)
Copied to clipboard
| Challenge: | Cross-lingual word embeddings are representations of words across languages in a shared continuous vector space. |
| Approach: | They propose a multilingual word embedding corpus which is acquired by neural machine translation and is based on monolingual data. |
| Outcome: | The proposed method is competitive with existing methods but on the cross-lingual document classification task, it obtains the best figures. |
Wikipedia2Vec: An Efficient Toolkit for Learning and Visualizing the Embeddings of Words and Entities from Wikipedia (2020.emnlp-demos)
Copied to clipboard
Ikuya Yamada, Akari Asai, Jin Sakuma, Hiroyuki Shindo, Hideaki Takeda, Yoshiyasu Takefuji, Yuji Matsumoto
| Challenge: | Existing tools for learning the embeddings of words and entities from Wikipedia are not yet available. |
| Approach: | They propose a Python-based tool for learning Wikipedia embeddings from Wikipedia . they use a Wikipedia dump file as an argument to issue a single command . |
| Outcome: | The proposed tool achieves state-of-the-art results on the KORE entity relatedness dataset and competitive results on benchmark datasets. |
Analyzing the Limitations of Cross-lingual Word Embedding Mappings (P19-1)
Copied to clipboard
| Challenge: | Existing methods for cross-lingual word embeddings have limited results . existing methods require little or no cross-linguistic signal to work . |
| Approach: | They compare offline mapping methods to an extension of skip-gram that jointly learns both embedding spaces. |
| Outcome: | The proposed method yields more isomorphic embeddings, is less sensitive to hubness, and achieves stronger results in bilingual lexicon induction. |