Non-Autoregressive Neural Machine Translation: A Call for Clarity (2022.emnlp-main)
Copied to clipboard
| Challenge: | Non-autoregressive translation models require a single forward pass to generate the output sequence instead of iteratively producing each predicted token. |
| Approach: | They propose to use a single forward pass to generate the output sequence instead of iteratively producing each predicted token. |
| Outcome: | The proposed models improve translation quality and speed under third-party testing environments. |
Similar Papers
Fully Non-autoregressive Neural Machine Translation: Tricks of the Trade (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing non-autoregressive neural machine translation models are slow to learn the dependency between output tokens. |
| Approach: | They propose to use fully non-autoregressive neural machine translation (NAT) to predict tokens with single forward of neural networks. |
| Outcome: | The proposed model achieves state-of-the-art results on three translation benchmarks with comparable performance to autoregressive and iterative NAT systems. |
What Have We Achieved on Non-autoregressive Translation? (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have shown that non-autoregressive (NAT) methods underperform autoregressive methods (AT) however, their evaluation using BLEU has been shown to weakly correlate with human annotations. |
| Approach: | They propose to evaluate four representative NAT methods using BLEU to narrow the performance gap between autoregressive and autoregressive translations. |
| Outcome: | The proposed methods underperform NAT and autoregressive methods under more reliable evaluation metrics. |
Non-Autoregressive Machine Translation: It’s Not as Fast as it Seems (2022.naacl-main)
Copied to clipboard
| Challenge: | Efficient machine translation models are commercially important as they can increase inference speeds, reduce costs and carbon emissions. |
| Approach: | They compare NAR models with autoregressive models to evaluate their performance . they point out flaws in evaluation methodology and argue for consistent evaluation . |
| Outcome: | The proposed model is faster on GPUs, but slower under more realistic usage conditions. |
Improving Non-Autoregressive Neural Machine Translation via Modeling Localness (2022.coling-1)
Copied to clipboard
| Challenge: | Existing non-autoregressive neural machine translation models suffer from poor localization quality due to sequential dependencies within the target sentence. |
| Approach: | They propose to introduce local information into NAT models by explicitly introducing local information about surrounding words into the encoder and decoder sides to achieve localness-aware representations. |
| Outcome: | The proposed method can achieve significant improvements over strong NAT baselines. |
Hybrid-Regressive Paradigm for Accurate and Speed-Robust Neural Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Autoregressive translation (NAT) is less robust in decoding batch size and hardware settings than NAT. |
| Approach: | They propose a two-stage translation prototype that prompts a small number of AT predictions and fills in previously skipped tokens at once. |
| Outcome: | The proposed translation prototype achieves comparable translation quality with AT while having 1.5x faster inference speed regardless of batch size and device. |
Non-Autoregressive Machine Translation with Latent Alignments (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing non-autoregressive machine translation methods are lacking in the field of latent alignments. |
| Approach: | They propose two strong methods for non-autoregressive machine translation that model latent alignments with dynamic programming. |
| Outcome: | The proposed models achieve state-of-the-art on the WMT’14 EnDe task, compared with the autoregressive Transformer baseline. |
Enriching Non-Autoregressive Transformer with Syntactic and Semantic Structures for Neural Machine Translation (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing non-autoregressive models have boosted the efficiency of neural machine translation, but their performance is significantly worse than that of autoregressive counterparts. |
| Approach: | They propose to incorporate syntactic and semantic structures among natural languages into a non-autoregressive Transformer for the task of neural machine translation. |
| Outcome: | The proposed model achieves faster speed and keeps translation quality compared with other models. |
Integrating Translation Memories into Non-Autoregressive Machine Translation (2023.eacl-main)
Copied to clipboard
| Challenge: | Non-autoregressive machine translation (NAT) has made great progress, but most studies focus on standard translation tasks. |
| Approach: | They propose to train an edit-based NAT model with a Translation Memory (TM) they propose to modify the data presentation and introduce an extra deletion operation to reduce decoding load. |
| Outcome: | The proposed model performs on par with an autoregressive approach while reducing the decoding load. |
Hint-Based Training for Non-Autoregressive Machine Translation (D19-1)
Copied to clipboard
| Challenge: | AutoRegressive Translation models have to generate tokens sequentially during decoding and thus suffer from high inference latency. |
| Approach: | They propose to use hidden states and word alignments to help train NART models. |
| Outcome: | The proposed model improves on the WMT14 En-De and De-En datasets but is faster in inference than the current models. |
Improving Non-autoregressive Neural Machine Translation with Monolingual Data (2020.acl-main)
Copied to clipboard
| Challenge: | Neural machine translation is usually done via knowledge distillation from an autoregressive (AR) model. |
| Approach: | They leverage large monolingual corpora to improve the NAR model's performance by transferring the autoregressive model' s generalization ability while preventing overfitting. |
| Outcome: | The proposed methods on the WMT14 En-De and WMT16 En-Ro news translation tasks show that monolingual data augmentation improves the NAR model to approach the teacher AR model’s performance. |