| Challenge: | Existing approaches to machine translation support autoregressive, semi-autoregressive and refinement-based non-auto-regressives. |
| Approach: | They propose a unified approach for supporting different generation manners of machine translation including autoregressive, semi-autoregressive and refinement-based non-auto-regressives. |
| Outcome: | The proposed approach achieves better or competitive translation performance compared with strong baseline models in all the settings. |
Similar Papers
Non-Autoregressive Machine Translation with Latent Alignments (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing non-autoregressive machine translation methods are lacking in the field of latent alignments. |
| Approach: | They propose two strong methods for non-autoregressive machine translation that model latent alignments with dynamic programming. |
| Outcome: | The proposed models achieve state-of-the-art on the WMT’14 EnDe task, compared with the autoregressive Transformer baseline. |
Non-Autoregressive Neural Machine Translation: A Call for Clarity (2022.emnlp-main)
Copied to clipboard
| Challenge: | Non-autoregressive translation models require a single forward pass to generate the output sequence instead of iteratively producing each predicted token. |
| Approach: | They propose to use a single forward pass to generate the output sequence instead of iteratively producing each predicted token. |
| Outcome: | The proposed models improve translation quality and speed under third-party testing environments. |
Fully Non-autoregressive Neural Machine Translation: Tricks of the Trade (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing non-autoregressive neural machine translation models are slow to learn the dependency between output tokens. |
| Approach: | They propose to use fully non-autoregressive neural machine translation (NAT) to predict tokens with single forward of neural networks. |
| Outcome: | The proposed model achieves state-of-the-art results on three translation benchmarks with comparable performance to autoregressive and iterative NAT systems. |
Integrating Translation Memories into Non-Autoregressive Machine Translation (2023.eacl-main)
Copied to clipboard
| Challenge: | Non-autoregressive machine translation (NAT) has made great progress, but most studies focus on standard translation tasks. |
| Approach: | They propose to train an edit-based NAT model with a Translation Memory (TM) they propose to modify the data presentation and introduce an extra deletion operation to reduce decoding load. |
| Outcome: | The proposed model performs on par with an autoregressive approach while reducing the decoding load. |
Phrase-Based & Neural Unsupervised Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Recent advances in machine translation have reported near human-level performance on several languages, yet their effectiveness strongly relies on the availability of large amounts of parallel sentences. |
| Approach: | They propose two models that leverage a careful initialization of the parameters and denoising effect of language models. |
| Outcome: | The proposed models outperform the current methods on English-French and German-English benchmarks while being simpler and having fewer hyper-parameters. |
Synchronous Refinement for Neural Machine Translation (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to decode target sentences face a one-pass issue . generated wrong words are added to the historical context to affect the generation of subsequent target words, which hinders the performance of machine translation. |
| Approach: | They propose a synchronous refinement method to revise potential errors in the generated words by considering part of the target future context. |
| Outcome: | The proposed method can refine generated target words and generate the next target word synchronously. |
Token-Level Self-Evolution Training for Sequence-to-Sequence Learning (2023.acl-short)
Copied to clipboard
| Challenge: | Adaptive training approaches do not consider the variation of learning difficulty in different training steps, making the learning deterministic and sub-optimal. |
| Approach: | They propose a dynamic token-level self-evolution training method that reweighs the training losses of different target tokens based on priors. |
| Outcome: | Empirically, the proposed method yields significant improvements on three translation tasks. |
Hybrid-Regressive Paradigm for Accurate and Speed-Robust Neural Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Autoregressive translation (NAT) is less robust in decoding batch size and hardware settings than NAT. |
| Approach: | They propose a two-stage translation prototype that prompts a small number of AT predictions and fills in previously skipped tokens at once. |
| Outcome: | The proposed translation prototype achieves comparable translation quality with AT while having 1.5x faster inference speed regardless of batch size and device. |
Syntactically Supervised Transformers for Faster Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Standard decoders for neural machine translation generate a single token per timestep, which slows inference . a series of controlled experiments demonstrates that SynST decodes sentences 5x faster than the baseline autoregressive Transformer. |
| Approach: | They propose a syntactically supervised Transformer that generates all target tokens in one shot . synST is a variant of the Transformer architecture that autoregressively predicts a chunked parse tree . |
| Outcome: | The proposed method decodes sentences 5x faster than the baseline method on En-De and En-Fr datasets while achieving higher BLEU scores. |
Helping the Weak Makes You Strong: Simple Multi-Task Learning Improves Non-Autoregressive Translators (2022.emnlp-main)
Copied to clipboard
| Challenge: | Non-autoregressive (NAR) neural machine translation models require a conditional independence assumption on target sequences, resulting in less informative learning signals. |
| Approach: | They propose a model-agnostic multi-task learning framework to provide more informative learning signals for NAR models under conventional MLE training. |
| Outcome: | The proposed framework improves accuracy of multiple NAR baselines without additional decoding overhead. |