Improving Language and Modality Transfer in Translation by Character-level Modeling (2025.acl-long)
Copied to clipboard
| Challenge: | Current translation systems cover only 5% of the world's languages . expanding to the long-tail of low-resource languages requires data-efficient methods that rely on cross-lingual and cross-modal knowledge transfer. |
| Approach: | They propose a character-based approach to improve adaptability to new languages and modalities by using a teacher-student approach and parallel translation data to obtain a SONAR character-level encoder. |
| Outcome: | The proposed model outperforms subword-based models in speech-to-text translation on the FLEURS benchmark on 33 languages and achieves state-of-the-art generalizability to unseen languages. |
Similar Papers
To Translate or Not to Translate: A Systematic Investigation of Translation-Based Cross-Lingual Transfer to Low-Resource Languages (2024.naacl-long)
Copied to clipboard
| Challenge: | XLT with multilingual language models is superfluous, says a new study . mBERT, XLM-R and mT5 are effective for cross-lingual transfer, authors say . |
| Approach: | They propose to use multilingual language models to improve cross-lingual transfer (XLT) they propose to add reliable translations to training data for XLT even for non-MT languages . |
| Outcome: | The proposed approaches outperform zero-shot XLT with mLMs, the authors show . the authors believe their findings warrant a broader inclusion of more robust translation-based baselines in XL research. |
Low-resource Neural Machine Translation with Cross-modal Alignment (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing neural machine translation techniques rely on large monolingual corpus, which is costly for some low-resource languages. |
| Approach: | They propose a cross-modal contrastive learning method to learn a shared space for all languages by additional visual modality. |
| Outcome: | The proposed method can learn cross-modal and cross-lingual alignment with small amount of image-text pairs and achieves significant improvements over the text-only baseline. |
Multilingual Speech Translation from Efficient Finetuning of Pretrained Models (2021.acl-long)
Copied to clipboard
Xian Li, Changhan Wang, Yun Tang, Chau Tran, Yuqing Tang, Juan Pino, Alexei Baevski, Alexis Conneau, Michael Auli
| Challenge: | Recent advances in text pretraining and finetuning have improved multitasking applications significantly. |
| Approach: | They propose a minimalistic LNA finetuning approach to build multilingual speech-to-text translation using a pretrained speech encoder and text decoder. |
| Outcome: | The proposed approach surpasses the cascaded ST benchmark for 36 translation directions on the large-scale multilingual ST benchmark CoVoST 2. |
Few-Shot Learning Translation from New Languages (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows strong transfer learning capability to unseen languages in sequence-to-sequence neural networks . current transfer learning methods require much less downstream task data than would otherwise be required. |
| Approach: | They first train word embeddings models on varying amounts of data and plug them into a machine translation model. |
| Outcome: | The proposed model can learn Flores with only 500 parallel sentences and 31,250 sentences of monolingual data, and it can exceed 15 BLEU on unseen languages. |
A Survey of Multilingual Models for Automatic Speech Recognition (2022.lrec-1)
Copied to clipboard
| Challenge: | Automatic Speech Recognition (ASR) systems have achieved human-like performance for a few languages, but the majority of the world’s languages do not have usable systems due to the lack of large speech datasets to train these models. |
| Approach: | They propose to use unlabeled speech data to build multilingual ASR models that can be used for improved performance on low-resource languages. |
| Outcome: | The proposed models can be used to improve performance on low-resource languages by using unlabeled speech data. |
Cross-Lingual Transfer Learning for Speech Translation (2025.naacl-short)
Copied to clipboard
| Challenge: | Increasing interest in building multilingual foundation models for NLP and speech research has led to limited data collection for training ST systems. |
| Approach: | They propose to use Whisper to explore the behavior of multilingual speech foundation models with restricted data. |
| Outcome: | The proposed model can translate to Chinese with a single language, and it can perform transcriptions in other languages. |
Compact and Robust Models for Japanese-English Character-level Machine Translation (D19-52)
Copied to clipboard
| Challenge: | In recent years, neural machine translation (NMT) has made a great progress, and its translation quality has far surpassed the conventional statistical machine translation. |
| Approach: | They propose a character-level translation model which is mid-gated and multi-attention model for Japanese-English translation and propose to train them using a relatively narrow beam of width 4 or 5 . |
| Outcome: | The proposed models can translate the word containing Katakana by coining out a close word, and the model can produce tolerable results for noised sentences. |
Character-Level Translation with Self-attention (2020.acl-main)
Copied to clipboard
| Challenge: | Existing models for character-level neural machine translation operate on word-level, which makes them memory inefficient because of large vocabulary sizes. |
| Approach: | They propose a transformer-based model and a novel variant that uses convolutions to combine information from nearby characters to facilitate character interactions. |
| Outcome: | The proposed model outperforms the standard transformer model and learns more robust character alignments on bilingual and multilingual translation datasets. |
Multilingual Pixel Representations for Translation and Effective Cross-lingual Transfer (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that pixel representations can be finetuned across scripts without vocabulary extensions, adapters, or transliteration. |
| Approach: | They propose to use pixel representations to train multilingual machine translation models . they explore parameter sharing within and across scripts to better understand where they lead to positive transfer . |
| Outcome: | The proposed model improves on two multilingual datasets with different language coverage compared to subword embeddings . the proposed model can be finetuned cross-lingually or to unseen scripts, and is more data-efficient than other alternatives such as vocabulary expansion . |
Adapting Multilingual Models for Code-Mixed Translation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work has addressed the lack of gold standard code-mixed to pure language parallel data with data augmentation techniques. |
| Approach: | They propose a back-translation-based training scheme for code-mixed translation which eliminates dependence on external resources. |
| Outcome: | The proposed model beats previous work by up to +3.8 BLEU on code-mixed tasks. |