Papers by Alexandre Berard
Naver Labs Europe’s Systems for the Document-Level Generation and Translation Task at WNGT 2019 (D19-56)
Copied to clipboard
| Challenge: | Recent advances in machine translation and natural language generation have created many challenges in this field especially when context is considered. |
| Approach: | They propose to leverage data from machine translation and natural language generation tasks to do transfer learning between MT, NLG and MT with source-side metadata. |
| Outcome: | The proposed approach outperforms the previous state-of-the-art on the Rotowire NLG task. |
Monolingual Adapters for Zero-Shot Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing adapter layers are more parameter-efficient and provide better performance than bilingual ones. |
| Approach: | They propose to use monolingual adapter layers instead of bilingual ones to compose them and generalize to unseen language pairs. |
| Outcome: | The proposed adapter layer formalism achieves a median improvement of +2.77 BLEU points over a 20-language multilingual Transformer baseline trained on TED talks. |
Multilingual Unsupervised Neural Machine Translation with Denoising Adapters (2021.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual unsupervised machine translation is a computationally expensive and hard to tune approach . auxiliary parallel data is used to train translation systems from monolingual data . |
| Approach: | They propose to use auxiliary parallel language pairs to train unsupervised machine translations . they propose to add auxiliary languages to pre-trained mBART-50 models with denoising adapters . |
| Outcome: | The proposed approach is on-par with back-translation and allows adding unseen languages incrementally. |
What Do Compressed Multilingual Machine Translation Models Forget? (2022.findings-emnlp)
Copied to clipboard
Alireza Mohammadshahi, Vassilina Nikoulina, Alexandre Berard, Caroline Brun, James Henderson, Laurent Besacier
| Challenge: | Recent studies show that pre-trained models achieve state-of-the-art results in NLP tasks but their size makes it more challenging to apply them in resource-constrained environments. |
| Approach: | They assess the impact of compression methods on multilingual Neural Machine Translation models for various language groups, gender, and semantic biases. |
| Outcome: | The proposed compression methods improve models on different benchmarks for language groups, gender, and semantic biases. |
SMaLL-100: Introducing Shallow Multilingual Machine Translation Model for Low-Resource Languages (2022.emnlp-main)
Copied to clipboard
Alireza Mohammadshahi, Vassilina Nikoulina, Alexandre Berard, Caroline Brun, James Henderson, Laurent Besacier
| Challenge: | Existing models for multilingual machine translation use scaling up the number of parameters to overcome the curse of multilinguality. |
| Approach: | They propose a multilingual machine translation model that shares information between similar languages and scales up the number of parameters to overcome the curse of multilinguality. |
| Outcome: | The proposed model outperforms previous models on low-resource benchmarks while improving inference latency and memory usage. |
Machine Translation of Restaurant Reviews: New Corpus for Domain Adaptation and Robustness (D19-56)
Copied to clipboard
Alexandre Berard, Ioan Calapodescu, Marc Dymetman, Claude Roux, Jean-Luc Meunier, Vassilina Nikoulina
| Challenge: | BLEU: MT is a very robust and efficient way to translate user-generated content. |
| Approach: | They propose a task to encourage research on MT robustness and domain adaptation . they ask professionals to translate 11.5k french 4SQ reviews to English . |
| Outcome: | The proposed task improves on the existing MT systems in a real-world scenario . the proposed methods improve translation accuracy and sentiment analysis . |
Efficient Inference for Multilingual Neural Machine Translation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual NMT is an attractive solution for production, but to match bilingual quality, it comes at the cost of larger and slower models. |
| Approach: | They propose to use a shallow decoder with vocabulary filtering to speed up inference . they validate their findings with BLEU and chrF on 380 language pairs . |
| Outcome: | The proposed approach can be used in two 20-language multi-parallel settings. |
Understanding and Mitigating Language Confusion in LLMs (2024.emnlp-main)
Copied to clipboard
| Challenge: | Llama Instruct and Mistral models exhibit high degrees of language confusion and even the strongest models fail to consistently respond in the correct language. |
| Approach: | They develop a language confusion benchmark to evaluate LLMs' inability to consistently generate text in a user’s desired language. |
| Outcome: | The proposed model fails to consistently respond in the correct language, despite being prone to high temperatures and complex prompts. |
Memory-efficient NLLB-200: Language-specific Expert Pruning of a Massively Multilingual Machine Translation Model (2023.acl-long)
Copied to clipboard
| Challenge: | Neural Machine Translation models are based on a Mixture of Experts architecture and can be pruned to remove up to 80% of experts without further finetuning. |
| Approach: | They propose a pruning method that removes up to 80% of experts without further finetuning and with a negligible loss in translation quality. |
| Outcome: | The proposed pruning method removes up to 80% of experts without further finetuning and with a negligible loss in translation quality. |