Data Augmentation with Unsupervised Machine Translation Improves the Structural Similarity of Cross-lingual Word Embeddings (2021.acl-srw)
Copied to clipboard
| Challenge: | Unsupervised cross-lingual word embedding methods learn a linear transformation matrix that maps two monolingual embeddable spaces that are separately trained with monolingual corpora. |
| Approach: | They propose a method that maps two monolingual embedding spaces that are separately trained with monolingual corpora using a pseudo-parallel corpus. |
| Outcome: | The proposed method outperforms other methods given the same amount of data and shows that using a pseudo-parallel corpus makes the source and target corpora (partially) parallel . |
Similar Papers
Unsupervised Joint Training of Bilingual Word Embeddings (P19-1)
Copied to clipboard
| Challenge: | Existing methods for unsupervised bilingual word embeddings are limited by the dissimilarity between the word embedded spaces. |
| Approach: | They propose a method that trains unsupervised bilingual word embeddings jointly on parallel data generated through unsupervised machine translation. |
| Outcome: | The proposed method outperforms unsupervised mapped bilingual word embeddings in cross-lingual NLP tasks. |
Do We Really Need Fully Unsupervised Cross-Lingual Embeddings? (D19-1)
Copied to clipboard
| Challenge: | a series of bilingual lexicon induction experiments with 15 diverse languages (210 language pairs) show that fully unsupervised CLWE methods fail for a large number of language pairs. |
| Approach: | They propose to use fully unsupervised approaches to project monolingual embeddings into a shared cross-lingual space without any cross-linguistic signal. |
| Outcome: | The proposed methods fail for a large number of language pairs, but never surpass weakly supervised methods. |
Cross-Lingual Word Embedding Refinement by ℓ1 Norm Optimisation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for building high-quality CLWEs learn mappings that minimise the l2 norm loss function but this optimisation objective has been shown to be sensitive to outliers. |
| Approach: | They propose a simple post-processing step to improve cross-lingual word embeddings using the Manhattan norm goodness-of-fit criterion. |
| Outcome: | The proposed approach outperforms four state-of-the-art baselines in bilingual lexicon induction and cross-lingual transfer tasks. |
Bilingual Lexicon Induction through Unsupervised Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Existing methods for bilingual lexicon induction use nearest neighbor or related retrieval methods to induce word translation pairs. |
| Approach: | They propose a method that aligns word embeddings in two languages and uses them to build a phrase-table and a language model to extract the bilingual lexicon. |
| Outcome: | The proposed method improves accuracy 6 points over nearest neighbor and 4 points over CSLS retrieval on the same cross-lingual embeddings. |
A Simple Approach to Learning Unsupervised Multilingual Embeddings (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work on unsupervised cross-lingual embeddings in the bilingual setting has given the impetus to learning a shared embeddable space for several languages. |
| Approach: | They propose to solve two sub-problems together to learn a shared embedding space for several languages. |
| Outcome: | The proposed approach outperforms existing methods in bilingual lexicon induction, cross-lingual word similarity, multilingual document classification, and multilingual dependency parsing tasks. |
A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings (P18-1)
Copied to clipboard
| Challenge: | Existing methods to learn cross-lingual word embeddings have failed in more realistic scenarios . a fully unsupervised initialization and a robust self-learning algorithm are needed to improve the existing methods. |
| Approach: | They propose an unsupervised initialization method that exploits structural similarity of embeddings and a robust self-learning algorithm that iteratively improves it. |
| Outcome: | The proposed method achieves the best published results in standard datasets even surpassing previous supervised systems. |
CLUSE: Cross-Lingual Unsupervised Sense Embeddings (D18-1)
Copied to clipboard
| Challenge: | Existing models for learning bilingual sense embeddings that encode semantics are weak for recognizing multi-sense word representations . elucidation of word embeddables is difficult because they do not allow a word to have different meanings in different contexts. |
| Approach: | They propose a sense induction and representation learning model that learns bilingual sense embeddings that align well in the vector space. |
| Outcome: | The proposed model shows that the learned embeddings are aligned well in the vector space. |
Unsupervised Cross-lingual Transfer of Word Embedding Spaces (D18-1)
Copied to clipboard
| Challenge: | Existing methods for cross-lingual word mapping require cross-linguistic supervision, but this is not available for many low resource languages. |
| Approach: | They propose an unsupervised method that learns transformation functions over corresponding word embedding spaces using a distributed distributional matching algorithm. |
| Outcome: | The proposed method performs better on bilingual lexicon induction and cross-lingual word similarity prediction datasets than other supervised and unsupervised methods. |
Improving Unsupervised Word-by-Word Translation with Language Model and Denoising Autoencoder (D18-1)
Copied to clipboard
| Challenge: | Unsupervised learning of cross-lingual word embeddings has fundamental limitations in translating sentences. |
| Approach: | They propose a method to improve word-by-word translation of cross-lingual embeddings using monolingual corpora without any back-translation. |
| Outcome: | The proposed system surpasses state-of-the-art unsupervised translation systems without costly iterative training. |
Analyzing the Limitations of Cross-lingual Word Embedding Mappings (P19-1)
Copied to clipboard
| Challenge: | Existing methods for cross-lingual word embeddings have limited results . existing methods require little or no cross-linguistic signal to work . |
| Approach: | They compare offline mapping methods to an extension of skip-gram that jointly learns both embedding spaces. |
| Outcome: | The proposed method yields more isomorphic embeddings, is less sensitive to hubness, and achieves stronger results in bilingual lexicon induction. |