Papers by Marcin Junczys-Dowmunt
Minimally-Augmented Grammatical Error Correction (D19-55)
Copied to clipboard
| Challenge: | Existing approaches to automatic grammatical error correction require error-labelled training data to achieve their best performance. |
| Approach: | They propose an unsupervised method that generates noise from inverted spell-checkers by using a synthetic error generation method. |
| Outcome: | The proposed method outperforms the current state-of-the-art for German and Russian GEC tasks without using real error-labelled training data. |
Near Human-Level Performance in Grammatical Error Correction with Hybrid Machine Translation (N18-2)
Copied to clipboard
| Challenge: | Currently, most effective GEC systems are based on phrase-based statistical machine translation. |
| Approach: | They combine two of the most popular approaches to automated Grammatical Error Correction (GEC) they create a hybrid GEC system that preserves the accuracy of SMT output and generates more fluent sentences . |
| Outcome: | The proposed system achieves state-of-the-art on the CoNLL-2014 and JFLEG benchmarks. |
From Research to Production and Back: Ludicrously Fast Neural Machine Translation (D19-56)
Copied to clipboard
Young Jin Kim, Marcin Junczys-Dowmunt, Hany Hassan, Alham Fikri Aji, Kenneth Heafield, Roman Grundkiewicz, Nikolay Bogoychev
| Challenge: | Using the dominating submissions to the previous edition of the shared task, we develop improved teacher-student training via multi-agent dual-learning and noisy backward-forward translation for Transformer-based student models. |
| Approach: | They propose to use multi-agent dual-learning and noisy backward-forward translation to improve teacher-student training for Transformer-based student models. |
| Outcome: | The proposed model outperforms submissions to the previous edition of the WNGT efficiency shared task by 4 BLEU points and 10 BLUE points respectively. |
Accelerating Asynchronous Stochastic Gradient Descent for Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | In order to achieve faster training we increase the mini-batch size and scale the learning rate accordingly. |
| Approach: | They propose a technique that delays gradient updates by increasing the mini-batch size to improve the model's convergence. |
| Outcome: | The proposed technique can train a shallow machine translation system 27% faster than an optimized baseline with negligible penalty in BLEU. |
Marian: Fast Neural Machine Translation in C++ (P18-4)
Copied to clipboard
Marcin Junczys-Dowmunt, Roman Grundkiewicz, Tomasz Dwojak, Hieu Hoang, Kenneth Heafield, Tom Neckermann, Frank Seide, Ulrich Germann, Alham Fikri Aji, Nikolay Bogoychev, André F. T. Martins, Alexandra Birch
| Challenge: | In this paper, we present Marian, an efficient and self-contained Neural Machine Translation framework . Marian is written in pure C++ with minimal dependencies . |
| Approach: | They present Marian, an efficient and self-contained Neural Machine Translation framework written in pure C++ with minimal dependencies. |
| Outcome: | The proposed framework achieves high training and translation speed with minimal dependencies . it is currently being deployed in multiple European projects . |
Levenshtein Training for Word-level Quality Estimation (2021.emnlp-main)
Copied to clipboard
| Challenge: | a novel scheme to perform word-level quality estimation is proposed for word-based quality estimation . authors propose a two-stage transfer learning procedure on augmented and human data . a Levenshtein Transformer can learn to post-edit without explicit supervision. |
| Approach: | They propose a novel scheme to use a Levenshtein Transformer to perform word-level quality estimation. |
| Outcome: | The proposed method performs better under data-constrained and unconstrained conditions than existing methods. |
On-the-Fly Fusion of Large Language Models and Machine Translation (2024.findings-naacl)
Copied to clipboard
| Challenge: | a weaker-at-translation LLM can improve translations of a NMT model, compared to a strong dedicated model. |
| Approach: | They propose to ensemble a neural machine translation model with a large language model, prompted on the same task and input. |
| Outcome: | The proposed method can be combined with various techniques from LLM prompting, such as in context learning and translation context. |
Approaching Neural Grammatical Error Correction as a Low-Resource Machine Translation Task (N18-1)
Copied to clipboard
| Challenge: | Previously, neural methods in grammatical error correction did not reach state-of-the-art results compared to phrase-based statistical machine translation (SMT) systems that improve on results by SMT use their set-up as a backbone for more complex systems. |
| Approach: | They propose a set of model-independent methods for neural GEC that can be easily applied in most GEC settings. |
| Outcome: | The proposed methods outperform state-of-the-art neural GEC systems by 10% M2 on the CoNLL-2014 benchmark and 5.9% on the JFLEG test set. |
PyMarian: Fast Neural Machine Translation and Evaluation in Python (2024.emnlp-demo)
Copied to clipboard
| Challenge: | a Python interface to Marian NMT is available in PyPI via pip install pymarian . the interface provides a speedup factor of up to 7.8 the existing implementations . |
| Approach: | They propose a Python interface to Marian NMT, a C++-based training and inference toolkit for sequence-to-sequence models. |
| Outcome: | The proposed interface enables models trained with Marian to be connected to Python tools with a speedup factor of up to 7.8 the existing implementations. |
The Curious Case of Hallucinations in Neural Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) suffers from well known pathologies such as coverage, mistranslation of named entities, etc. |
| Approach: | They propose a theory that explains hallucinations under source perturbation and a method that generates hallucines under corpus-level noise without any source perturbations. |
| Outcome: | The proposed hypothesis is validated by a corpus-level noise analysis and is validateable in other datasets. |