Papers with UNMT
Advances and Challenges in Unsupervised Neural Machine Translation (2021.eacl-tutorials)
Copied to clipboard
| Challenge: | Unsupervised neural machine translation (UNMT) has achieved impressive results, but there are still several challenges for the technology. |
| Approach: | They present a framework for unsupervised neural machine translation (UNMT) they examine the latest progress and challenges of UNMT and examine how it holds up . |
| Outcome: | The proposed method has achieved impressive results but still faces challenges. |
Improving the Lexical Ability of Pretrained Language Models for Unsupervised Neural Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for unsupervised neural machine translation (UNMT) use cross-lingual pretraining to align the lexical- and high-level representations of two languages. |
| Approach: | They propose to use type-level cross-lingual subword embeddings to enhance the bilingual masked language model pretraining with lexical-level information to align the two languages. |
| Outcome: | Empirical results show that the method improves on UNMT (up to 4.5 BLEU) and bilingual lexicon induction compared to baseline models. |
On-the-fly Cross-lingual Masking for Multilingual Pre-training (2023.acl-long)
Copied to clipboard
| Challenge: | In multilingual pre-training, multilingual models only learn cross-linguality implicitly from isomorphic spaces formed by overlapping different language spaces due to the lack of explicit cross-linguistic forward pass. |
| Approach: | They propose a dynamic token-wise masking scheme for multilingual pre-training that uses a special token [C]x to replace a random token in the input sentence. |
| Outcome: | The proposed model improves the performance of UNMT models on De, Ro, Ne En. |
Exploiting Curriculum Learning in Unsupervised Neural Machine Translation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Experimental results show that the proposed method achieves consistent improvements with faster convergence speed. |
| Approach: | They propose a curriculum learning method to gradually utilize pseudo bi-texts based on their quality from multiple granularities. |
| Outcome: | The proposed method achieves consistent improvements with faster convergence speed on WMT 14 En-Fr, WMT14 En-De, and LDC En-Zh translation tasks. |
Domain Mismatch Doesn’t Always Prevent Cross-lingual Transfer Learning (2022.lrec-1)
Copied to clipboard
| Challenge: | Recent studies have reported that domain mismatch prevents cross-lingual transfer . UBLI and UNMT do not work well when underlying monolingual corpora come from different domains . |
| Approach: | They show that a simple initialization regimen can overcome domain mismatch in cross-lingual transfer . they pre-train word embeddings on concatenated domain-mismatched corpora and use them as initializations . |
| Outcome: | The initialization regimen can overcome the domain mismatch effect in cross-lingual transfer learning . the initializations were used for MUSE UBLI, UN Parallel UNMT, and the SemEval 2017 task . |
Unsupervised Bilingual Word Embedding Agreement for Unsupervised Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Unsupervised bilingual word embedding (UBWE) has helped unsupervised neural machine translation (UNMT) achieve remarkable results in several language pairs. |
| Approach: | They propose two methods that train UNMT with UBWE agreement . they propose to use UBwe to initialize word embedding in UNMT . |
| Outcome: | The proposed methods outperform conventional methods on several language pairs. |
Unsupervised Neural Machine Translation for Low-Resource Domains via Meta-Learning (2021.acl-long)
Copied to clipboard
| Challenge: | Unsupervised machine translation suffers from data-scarce domains, authors report . a meta-learning algorithm trains the model to adapt to another domain by utilizing only a small amount of training data. |
| Approach: | They propose a meta-learning algorithm that trains the model to adapt to another domain . their model surpasses a transfer learning-based approach by up to 2-3 BLEU scores . |
| Outcome: | The proposed algorithm outperforms a transfer learning-based approach by 2-3 BLEU scores . the proposed model outperformed previous models in the domain of unsupervised machine translation . |
Self-Training for Unsupervised Neural Machine Translation in Unbalanced Training Data Scenarios (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods that use monolingual corpora for translation are not suitable for low-resource languages such as Estonian. |
| Approach: | They propose unsupervised neural machine translation (UNMT) that relies on monolingual corpora to train a robust UNMT system and improve its performance. |
| Outcome: | The proposed methods outperform conventional UNMT systems on several language pairs. |
Knowledge Distillation for Multilingual Unsupervised Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Unsupervised neural machine translation (UNMT) can only translate between a single language pair and cannot produce translation results for multiple language pairs at the same time. |
| Approach: | They propose a method to translate between 13 languages using a single encoder and a decoder . they propose two knowledge distillation methods to further enhance multilingual UNMT performance . |
| Outcome: | The proposed method improves translation performance for all languages using multilingual data. |
Reference Language based Unsupervised Neural Machine Translation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to use a common language as an auxiliary for better translation have a long tradition in machine translation. |
| Approach: | They propose a reference language-based framework for unsupervised neural machine translation that uses only one auxiliary language as an auxiliary for better translation. |
| Outcome: | The proposed framework improves the quality of pivot translation over a baseline that uses only one auxiliary language. |
Robust Unsupervised Neural Machine Translation with Adversarial Denoising Training (2020.coling-main)
Copied to clipboard
| Challenge: | Unsupervised neural machine translation (UNMT) has attracted great interest in the machine translation community. |
| Approach: | They propose to explicitly take noisy data into consideration to improve the robustness of UNMT based systems. |
| Outcome: | The proposed methods significantly improved the robustness of the conventional UNMT systems in noisy scenarios. |
POMP: Probability-driven Meta-graph Prompter for LLMs in Low-resource Unsupervised Neural Machine Translation (2024.acl-long)
Copied to clipboard
| Challenge: | Low-resource languages (LRLs) face challenges in supervised neural machine translation due to limited parallel data. |
| Approach: | They propose a method that uses a dynamic graph to organize auxiliary languages in prompts to improve LRL translations. |
| Outcome: | The proposed method improves translation accuracy in low-resource languages (LRLs) using auxiliary language pairs and synthetic pseudo-parallel data. |
Improving Unsupervised Neural Machine Translation via Training Data Self-Correction (2024.lrec-main)
Copied to clipboard
| Challenge: | Unsupervised neural machine translation models can generate mistakes during training . however, the quality of pseudo-parallel sentences cannot be guaranteed . |
| Approach: | They propose a method to improve the quality of pseudo-parallel sentences . they use token-level translations to correct mis-translated tokens . |
| Outcome: | Empirical results show that the proposed method outperforms baselines on widely used datasets. |