Latent Group Dropout for Multilingual and Multidomain Machine Translation (2022.findings-naacl)
Copied to clipboard
| Challenge: | Multidomain and multilingual machine translation often rely on parameter sharing strategies, which are hardcoded in the network architecture, independent of the similarities between tasks. |
| Approach: | They propose a method to take advantage of similarities by using a latent-variable model and develop techniques to train this model end-to-end. |
| Outcome: | The proposed model improves translation performance without increasing the model size. |
Similar Papers
Multilingual Machine Translation with Hyper-Adapters (2022.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual machine translation suffers from negative interference across languages. |
| Approach: | They propose a rescaling fix that reduces the number of parameters and enables training larger hyper-networks. |
| Outcome: | The proposed approach outperforms regular adapters and achieves the same performance with 12 times less parameters. |
Revisiting Multi-Domain Machine Translation (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing approaches to handle multi-domain machine translation systems are lacking due to the variability of data. |
| Approach: | They propose to use domain adaptation methods to handle situations where a sample of matched sentences is available in training and where only samples of source-side sentences are available. |
| Outcome: | The proposed model is able to handle multiple domains and their expectations with respect to performance. |
Multilingual Machine Translation: Closing the Gap between Shared and Language-specific Encoder-Decoders (2021.eacl-main)
Copied to clipboard
| Challenge: | State-of-the-art multilingual machine translation relies on a universal encoder-decoder, which requires retraining the entire system to add new languages. |
| Approach: | They propose an encoder-decoder approach that can be extended to new languages by learning their corresponding modules. |
| Outcome: | The proposed approach outperforms the universal encoder-decoder by 3.28 BLEU points on average while allowing to add new languages without retraining the rest of the modules. |
Multilingual Neural Machine Translation (2020.coling-tutorials)
Copied to clipboard
| Challenge: | In this tutorial, we will cover the latest advances in NMT to enhance low-resource translation. |
| Approach: | They will cover the latest advances in NMT approaches that leverage multilingualism . they will focus on topics such as language divergence, transfer learning and pivoting . |
| Outcome: | This tutorial will cover the latest advances in NMT to enhance low-resource translation models. |
Multilingual Pixel Representations for Translation and Effective Cross-lingual Transfer (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that pixel representations can be finetuned across scripts without vocabulary extensions, adapters, or transliteration. |
| Approach: | They propose to use pixel representations to train multilingual machine translation models . they explore parameter sharing within and across scripts to better understand where they lead to positive transfer . |
| Outcome: | The proposed model improves on two multilingual datasets with different language coverage compared to subword embeddings . the proposed model can be finetuned cross-lingually or to unseen scripts, and is more data-efficient than other alternatives such as vocabulary expansion . |
m^4 Adapter: Multilingual Multi-Domain Adaptation for Machine Translation with a Meta-Adapter (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Multilingual neural machine translation models (MNMT) are effective on transferring knowledge between high-resource languages to low-resourced languages. |
| Approach: | They propose a multilingual multi-domain adapter which combines domain and language knowledge using meta-learning with adapters. |
| Outcome: | The proposed model outperforms other adapter methods in a domain shift and language pair translation task. |
Three Strategies to Improve One-to-Many Multilingual Translation (D18-1)
Copied to clipboard
| Challenge: | Existing studies show that one-to-many multilingual translation cannot perform on par with the individually trained models. |
| Approach: | They propose to exploit unique initial states for target languages and language-dependent positional embeddings to create hidden cells of the encoder to achieve comparable or even better performance than individually trained models. |
| Outcome: | The proposed methods achieve comparable or even better performance than the individually trained models. |
Tricks for Training Sparse Translation Models (2022.naacl-main)
Copied to clipboard
| Challenge: | Multitask learning with an unbalanced data distribution skews model learning towards high resource tasks. |
| Approach: | They propose to use a temperature heating mechanism and dense pre-training to mitigate this by training models with a fixed model capacity. |
| Outcome: | The proposed techniques improve performance on two multilingual translation benchmarks compared to BASELayers and Dense scaling baselines and in combination, more than 2x model convergence speed. |
Emerging Cross-lingual Structure in Pretrained Language Models (2020.acl-main)
Copied to clipboard
| Challenge: | Recent work has shown that multilingual pretraining works, but is unable to measure these effects. |
| Approach: | They propose to use multilingual masked language modeling to train a model on concatenated text from multiple languages to find universal latent symmetries in embedding spaces. |
| Outcome: | The proposed models can be trained on concatenated text from multiple languages without shared vocabulary or domain similarity. |
Parameter-Efficient Cross-lingual Transfer of Vision and Language Models via Translation-based Alignment (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing cross-lingual transfer methods that use labeled data and linguistic resources would consume excessive resources for a large number of languages. |
| Approach: | They propose a parameter-efficient cross-lingual transfer learning framework that utilizes a translation-based alignment method to mitigate multilingual disparities. |
| Outcome: | The proposed framework reduces disparities among languages and improves cross-lingual transfer results in low-resource scenarios while keeping and fine-tuning only a small number of parameters. |