Papers with mBART50
Exploring Enhanced Code-Switched Noising for Pretraining in Neural Machine Translation (2023.findings-eacl)
Copied to clipboard
| Challenge: | Multilingual pretraining approaches to denoise synthetic code-switched data have shown that they generate the noise using non-contextual, one-to-one word translations obtained from lexicons. |
| Approach: | They propose an approach where contextual, many-to-many word translations are generated using a ‘base’ NMT model. |
| Outcome: | The proposed approach improves on 3 different language families and shows that small models can perform better than massive models like mBART50 and mRASP2 . |
PrahokBART: A Pre-trained Sequence-to-Sequence Model for Khmer Natural Language Generation (2025.coling-main)
Copied to clipboard
| Challenge: | Pre-trained sequence-to-sequence models are typically pretrained on extensive raw text corpora and fine-tuned on task-specific data. |
| Approach: | They introduce a pre-trained sequence-to-sequence model trained from scratch for Khmer using carefully curated Khmer and English corpora. |
| Outcome: | The proposed model outperforms existing models on three generative tasks and is data-efficient and effective in enhancing performance across various natural language generation tasks. |
IndicBART: A Pre-trained Model for Indic Natural Language Generation (2022.findings-acl)
Copied to clipboard
| Challenge: | IndicBART is a multilingual, sequence-to-sequence pre-trained model focusing on 11 Indic languages and English. |
| Approach: | They present a multilingual sequence-to-sequence pre-trained model for Indic languages . they evaluate it on two NLG tasks: Neural Machine Translation and extreme summarization . |
| Outcome: | The proposed model performs well on low-resource translation scenarios . Script sharing, multilingual training, and better utilization contribute to the performance. |