Papers by David Grangier
Understanding Back-Translation at Scale (D18-1)
Copied to clipboard
| Challenge: | An effective method to improve neural machine translation with monolingual data is to augment the parallel training corpus with back-translations of target language sentences. |
| Approach: | They propose to augment parallel training corpus with back-translations of target language sentences to improve neural machine translation with monolingual data. |
| Outcome: | The proposed method achieves a state-of-the-art of 35 BLEU on the WMT’14 English-German test set. |
Assessing the Role of Data Quality in Training Bilingual Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that adding more languages can degrade performance for some languages while improving others. |
| Approach: | They propose a data filtering strategy to select high-quality bilingual training data with only high quality English data. |
| Outcome: | The proposed approach improves bilingual model performance by 2–4% and reduces bilingual models performance gaps to 1%. |
QuickEdit: Editing Text & Translations by Crossing Words Out (N18-1)
Copied to clipboard
| Challenge: | Using statistical learning, a computer can rephrase a sentence by only pointing at words that should be avoided. |
| Approach: | They propose a framework for computer-assisted text editing that relies on simple interactions between human editors and tokens. |
| Outcome: | The proposed framework allows to get substantial modifications to a sentence without human intervention. |
On Systematic Style Differences between Unsupervised and Supervised MT and an Application for High-Resource Machine Translation (2022.naacl-main)
Copied to clipboard
| Challenge: | Modern unsupervised machine translation systems reach reasonable translation quality under clean and controlled data conditions. |
| Approach: | They compare unsupervised and supervised machine translation systems of similar quality . they combine the benefits of both methods into a single system . |
| Outcome: | The proposed system improves adequacy and fluency as measured by human evaluators. |
fairseq: A Fast, Extensible Toolkit for Sequence Modeling (N19-4)
Copied to clipboard
Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, Michael Auli
| Challenge: | OpenNMT is a community-built toolkit written in multiple languages with an emphasis on extensibility. |
| Approach: | They propose to use PyTorch to train custom sequence models for translation, summarization, language modeling, and other tasks. |
| Outcome: | The proposed toolkit is fast, extensible, and useful for both research and production. |
A Natural Diet: Towards Improving Naturalness of Machine Translation Output (2022.findings-acl)
Copied to clipboard
| Challenge: | MT evaluation often focuses on accuracy and fluency without paying much attention to translation style. |
| Approach: | They propose a method for training machine translation systems to achieve a more natural style by contrasting training data according to the naturalness of the target side. |
| Outcome: | The proposed method achieves lexical richness on par with human translations, and is preferred by human experts when compared to baseline translations. |
High Quality Rather than High Model Probability: Minimum Bayes Risk Decoding with Neural Metrics (2022.tacl-1)
Copied to clipboard
| Challenge: | Neural machine translations are ranked below human translations in professional evaluations . |
| Approach: | They apply minimum bayes risk decoding to optimize different metrics of translation quality . they show that model estimates and translation quality only vaguely correlate . |
| Outcome: | The proposed method improves human translations with different models and metric. |
BLEU might be Guilty but References are not Innocent (2020.emnlp-main)
Copied to clipboard
| Challenge: | Using a method to collect references and compare their value with human evaluations, we show that multi-reference BLEU does not improve the correlation for high quality output. |
| Approach: | They propose a method to compare the quality of automated metrics by analyzing references and comparing them with human evaluations. |
| Outcome: | The proposed method improves correlation with all modern evaluation metrics including embedding-based methods. |
Efficient Content-Based Sparse Attention with Routing Transformers (2021.tacl-1)
Copied to clipboard
| Challenge: | Self-attention suffers from quadratic computation and memory requirements with respect to sequence length . despite its effectiveness, self-attention models suffer from quadratic computation and a limited set of locations . |
| Approach: | They propose to learn dynamic sparse attention patterns that avoid allocating computation and memory to attend to content unrelated to the query of interest. |
| Outcome: | The proposed model outperforms similar sparse attention models on language modeling and image generation on Wikitext-103 . |
Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modeling (2024.acl-long)
Copied to clipboard
| Challenge: | Large language model pre-training is infeasible due to the large compute costs and duration associated with pre- training and the impending scarcity of high-quality data on the web. |
| Approach: | They propose to use an off-the-shelf instruction-tuned model prompted to paraphrase documents on the web in specific styles such as “like Wikipedia” or in “question-answer format” to jointly pre-train LLMs on real and synthetic rephrases. |
| Outcome: | The proposed model speeds up pre-training by 3x on the C4 dataset, and improves perplexity by 50% on average across different subsets of the Pile. |
Classical Structured Prediction Losses for Sequence to Sequence Learning (N18-1)
Copied to clipboard
| Challenge: | Recent work on training neural attention models at the sequence level has focused on a series of objective functions commonly used for structured prediction. |
| Approach: | They propose to use objective functions commonly used to train linear models for structured prediction to train neural attention models at the sequence-level using either reinforcement learning-style methods or beam search optimization. |
| Outcome: | The proposed model outperforms beam search optimization on German-English translation and abstractive summarization tasks. |
Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation (2021.tacl-1)
Copied to clipboard
| Challenge: | a large study of machine translation systems shows poor evaluation procedures can lead to erroneous conclusions. |
| Approach: | They propose an evaluation methodology grounded in explicit error analysis based on the Multidimensional Quality Metrics framework. |
| Outcome: | The proposed evaluation methodology outperforms crowd workers in two languages . it shows that human-based metrics outperformed crowd workers . |
Training Bilingual LMs with Data Constraints in the Targeted Language (2025.findings-acl)
Copied to clipboard
| Challenge: | a large number of languages have insufficient data for pretraining, but most non-English models are trained on scrapes of the web. |
| Approach: | They propose to use data from an auxiliary language to boost model performance . they quantify the performance gap between training with data in a data-rich auxiliary and training in the target language . |
| Outcome: | The proposed method boosts model performance in a target language with insufficient data . it also explores the benefits of translation systems and the limitations of model scaling when data is limited. |
Toward Better Storylines with Sentence-Level Language Models (2020.acl-main)
Copied to clipboard
| Challenge: | Rather than modeling fluency, the sentence-level language model can focus on longer range dependencies, which are crucial for multi-sentence coherence. |
| Approach: | They propose a sentence-level language model which selects the next sentence in a story from a finite set of fluent alternatives. |
| Outcome: | The proposed model can focus on longer range dependencies, crucial for multi-sentence coherence. |
Unsupervised Paraphrasing without Translation (P19-1)
Copied to clipboard
| Challenge: | Recent work on automatic paraphrasing focuses on methods leveraging machine translation as an intermediate step. |
| Approach: | They propose to learn paraphrasing models only from a monolingual corpus . they propose a residual variant of vector-quantized variational auto-encoder . |
| Outcome: | The proposed model outperforms supervised and unsupervised translation methods in paraphrase identification and training set augmentation. |
Translationese as a Language in “Multilingual” NMT (2020.acl-main)
Copied to clipboard
| Challenge: | Recent work examines the impact of translationese in machine translation evaluation using the WMT evaluation campaign. |
| Approach: | They propose to use a sentence-level classifier to distinguish translationese from original target text to generate a machine translation model that can produce more natural outputs at test time. |
| Outcome: | The proposed model produces more natural outputs at test time, yielding gains in human evaluation scores on accuracy and fluency. |
ELI5: Long Form Question Answering (P19-1)
Copied to clipboard
| Challenge: | Existing question answering datasets provide extractive or short answers, but less attention has been paid to open-ended questions that require explanations. |
| Approach: | They present a large-scale corpus for long form question answering . they use a Reddit forum to provide elaborate answers to open-ended questions . |
| Outcome: | The proposed model outperforms Seq2Seq, language modeling, and other models in human evaluations. |
The Trade-offs of Domain Adaptation for Neural Language Models (2022.acl-long)
Copied to clipboard
| Challenge: | Neural Language Models (LMs) trained on large generic training sets have been shown to be effective at adapting to smaller, specific target domains for language modeling and other downstream tasks. |
| Approach: | They propose a framework for a Neural Language Models (LM) to be presented in a common framework. |
| Outcome: | The proposed framework highlights similarities and subtle differences between adaptation techniques and the framework. |