Unsupervised Multilingual Word Embedding with Limited Resources using Neural Language Models (P19-1)
Copied to clipboard
| Challenge: | Existing methods that map word embeddings into a common space without any parallel data or pre-training have been proposed that are limited in resources and perform poorly under resource-poor conditions. |
| Approach: | They propose a model that maps monolingual word embeddings into a common space without any parallel data and generates multilingual embeddables without any pre-training. |
| Outcome: | The proposed model outperforms existing methods on word alignment tasks on low-resource conditions and with limited resources. |
Similar Papers
A Simple Approach to Learning Unsupervised Multilingual Embeddings (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work on unsupervised cross-lingual embeddings in the bilingual setting has given the impetus to learning a shared embeddable space for several languages. |
| Approach: | They propose to solve two sub-problems together to learn a shared embedding space for several languages. |
| Outcome: | The proposed approach outperforms existing methods in bilingual lexicon induction, cross-lingual word similarity, multilingual document classification, and multilingual dependency parsing tasks. |
Learning Unsupervised Multilingual Word Embeddings with Incremental Multilingual Hubs (N19-1)
Copied to clipboard
| Challenge: | Recent research has found that a shared bilingual word embedding space can be induced by projecting monolingual word embeds from two languages without any bilingual supervision. |
| Approach: | They propose a framework for learning unsupervised multilingual word embeddings that mitigates instability issues for distant language pairs. |
| Outcome: | The proposed framework outperforms the state-of-the-art methods on two downstream tasks outperforming even supervised baselines. |
Unsupervised Cross-lingual Transfer of Word Embedding Spaces (D18-1)
Copied to clipboard
| Challenge: | Existing methods for cross-lingual word mapping require cross-linguistic supervision, but this is not available for many low resource languages. |
| Approach: | They propose an unsupervised method that learns transformation functions over corresponding word embedding spaces using a distributed distributional matching algorithm. |
| Outcome: | The proposed method performs better on bilingual lexicon induction and cross-lingual word similarity prediction datasets than other supervised and unsupervised methods. |
Improving Low-Resource Languages in Pre-Trained Multilingual Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained multilingual language models are the foundation of many NLP approaches, but are often not well-supported by these models due to small available monolingual corpora. |
| Approach: | They propose an unsupervised approach to improve cross-lingual representations of low-resource languages by bootstrapping word translation pairs from monolingual corpora and using them to improve language alignment. |
| Outcome: | The proposed approach improves cross-lingual representations on low-resource languages using word retrieval and zero-shot named entity recognition. |
Unsupervised Joint Training of Bilingual Word Embeddings (P19-1)
Copied to clipboard
| Challenge: | Existing methods for unsupervised bilingual word embeddings are limited by the dissimilarity between the word embedded spaces. |
| Approach: | They propose a method that trains unsupervised bilingual word embeddings jointly on parallel data generated through unsupervised machine translation. |
| Outcome: | The proposed method outperforms unsupervised mapped bilingual word embeddings in cross-lingual NLP tasks. |
Unsupervised Multilingual Sentence Embeddings for Parallel Corpus Mining (2020.acl-srw)
Copied to clipboard
| Challenge: | Existing models of multilingual sentence embeddings require large parallel data resources which are not available for low-resource languages. |
| Approach: | They propose an unsupervised method to derive multilingual sentence embeddings using monolingual data. |
| Outcome: | The proposed method improves on two parallel corpus mining tasks and for other languages. |
Unsupervised Cross-Lingual Representation Learning (P19-4)
Copied to clipboard
| Challenge: | a comprehensive survey of cutting-edge weakly-supervised and unsupervised cross-lingual word representations is presented . |
| Approach: | This tutorial provides a comprehensive survey of recent work on weakly-supervised and unsupervised cross-lingual word representations. |
| Outcome: | This tutorial provides a comprehensive survey of cutting-edge weakly-supervised and unsupervised word representations. |
Unsupervised Multilingual Word Embeddings (D18-1)
Copied to clipboard
| Challenge: | Prior art for learning UMWEs relies on a number of independently trained UBWEs to obtain multilingual embeddings. |
| Approach: | They propose a fully unsupervised framework that exploits the relations between all language pairs to learn multilingual embeddings without cross-lingual supervision. |
| Outcome: | The proposed framework outperforms supervised approaches on multilingual word translation and cross-lingual word similarity and beats a number of other approaches trained with cross-linguistic resources. |
A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings (P18-1)
Copied to clipboard
| Challenge: | Existing methods to learn cross-lingual word embeddings have failed in more realistic scenarios . a fully unsupervised initialization and a robust self-learning algorithm are needed to improve the existing methods. |
| Approach: | They propose an unsupervised initialization method that exploits structural similarity of embeddings and a robust self-learning algorithm that iteratively improves it. |
| Outcome: | The proposed method achieves the best published results in standard datasets even surpassing previous supervised systems. |
KIT-Multi: A Translation-Oriented Multilingual Embedding Corpus (L18-1)
Copied to clipboard
| Challenge: | Cross-lingual word embeddings are representations of words across languages in a shared continuous vector space. |
| Approach: | They propose a multilingual word embedding corpus which is acquired by neural machine translation and is based on monolingual data. |
| Outcome: | The proposed method is competitive with existing methods but on the cross-lingual document classification task, it obtains the best figures. |