Papers by Vedanuj Goswami
Language-Aware Multilingual Machine Translation with Self-Supervised Learning (2023.findings-eacl)
Copied to clipboard
| Challenge: | Multilingual machine translation (MMT) is a challenging multitask optimization problem because of lack of a framework to learn language-specific parameters. |
| Approach: | They propose a self-supervised learning task that denies monolingual data to MMT . they then propose 'intra-distillation' task that co-trains with MMT task . |
| Outcome: | The proposed approach outperforms three state-of-the-art methods on 8-language and 15-language benchmarks. |
SpeechMatrix: A Large-Scale Mined Corpus of Multilingual Speech-to-Speech Translations (2023.acl-long)
Copied to clipboard
Paul-Ambroise Duquenne, Hongyu Gong, Ning Dong, Jingfei Du, Ann Lee, Vedanuj Goswami, Changhan Wang, Juan Pino, Benoît Sagot, Holger Schwenk
| Challenge: | SpeechMatrix is a large-scale multilingual corpus of speech-to-speech translations mined from real speech of European Parliament recordings. |
| Approach: | They present a large-scale multilingual corpus of speech-to-speech translations mined from real speech of European Parliament recordings. |
| Outcome: | The proposed model can train bilingual models on 136 language pairs with 418 thousand hours of speech. |
Causes and Cures for Interference in Multilingual Translation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods to reduce interference in multilingual machine translation are often computationally intensive and do not always work. |
| Approach: | They propose to reduce interference in multilingual machine translation models by enlarging the model and tuning the sampling temperature to control the proportion of each language pair in the data. |
| Outcome: | The proposed model size, data size, and proportion of each language pair within the dataset determine interference (or synergy) . |
Towards Being Parameter-Efficient: A Stratified Sparsely Activated Transformer with Dynamic Capacity (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have established that Mixture-of-experts models are parameter-inefficient as the improvement in performance diminishes with an increasing number of experts. |
| Approach: | They propose a mix-of-experts model that uses sparse activation to increase the number of parameters while maintaining low computational requirements per token. |
| Outcome: | The proposed models outperform state-of-the-art models on three multilingual machine translation benchmarks with 4, 15, and 94 language pairs. |
Revisiting Machine Translation for Cross-lingual Classification (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent work in cross-lingual learning has pivoted around multilingual models, which are typically pretrained on unlabeled corpora in multiple languages using some form of language modeling objective. |
| Approach: | They propose to use a stronger machine translation system to mitigat mismatch between training on original text and running inference on machine translated text. |
| Outcome: | The proposed approach is highly task dependent and calls into question the dominance of multilingual models for cross-lingual classification. |
Tricks for Training Sparse Translation Models (2022.naacl-main)
Copied to clipboard
| Challenge: | Multitask learning with an unbalanced data distribution skews model learning towards high resource tasks. |
| Approach: | They propose to use a temperature heating mechanism and dense pre-training to mitigate this by training models with a fixed model capacity. |
| Outcome: | The proposed techniques improve performance on two multilingual translation benchmarks compared to BASELayers and Dense scaling baselines and in combination, more than 2x model convergence speed. |
Small Data, Big Impact: Leveraging Minimal Data for Effective Machine Translation (2023.acl-long)
Copied to clipboard
Jean Maillard, Cynthia Gao, Elahe Kalbassi, Kaushik Ram Sadagopan, Vedanuj Goswami, Philipp Koehn, Angela Fan, Francisco Guzman
| Challenge: | Existing datasets are not economical to create large-scale datasets, but for low-resource languages, a few thousand professionally translated sentence pairs can be useful. |
| Approach: | They propose to use a dataset to train machine translation models on pre-existing and synthetic data to augment them with millions of sentences through backtranslation. |
| Outcome: | The proposed model can cover hundreds of languages with high quality training data even when smaller but lower quality datasets are used. |