| Challenge: | Existing non-autoregressive translation models struggle with document context and handling discourse phenomena. |
| Approach: | They propose a simple but effective design of sentence alignment between source and target to improve their performance on document-level machine translation. |
| Outcome: | The proposed model achieves high acceleration on documents and sentence alignment significantly enhances their performance. |
Similar Papers
What Have We Achieved on Non-autoregressive Translation? (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have shown that non-autoregressive (NAT) methods underperform autoregressive methods (AT) however, their evaluation using BLEU has been shown to weakly correlate with human annotations. |
| Approach: | They propose to evaluate four representative NAT methods using BLEU to narrow the performance gap between autoregressive and autoregressive translations. |
| Outcome: | The proposed methods underperform NAT and autoregressive methods under more reliable evaluation metrics. |
Improving Non-Autoregressive Neural Machine Translation via Modeling Localness (2022.coling-1)
Copied to clipboard
| Challenge: | Existing non-autoregressive neural machine translation models suffer from poor localization quality due to sequential dependencies within the target sentence. |
| Approach: | They propose to introduce local information into NAT models by explicitly introducing local information about surrounding words into the encoder and decoder sides to achieve localness-aware representations. |
| Outcome: | The proposed method can achieve significant improvements over strong NAT baselines. |
Revisiting Non-Autoregressive Translation at Scale (2023.findings-acl)
Copied to clipboard
| Challenge: | Extensive experiments on two advanced NAT models show scaling can improve translation performance. |
| Approach: | They empirically examine the impact of scaling on NAT behaviors on a large-scale WMT dataset. |
| Outcome: | The proposed model can achieve comparable performance with the scaling model while maintaining the superiority of decoding speed with standard NAT models. |
Fully Non-autoregressive Neural Machine Translation: Tricks of the Trade (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing non-autoregressive neural machine translation models are slow to learn the dependency between output tokens. |
| Approach: | They propose to use fully non-autoregressive neural machine translation (NAT) to predict tokens with single forward of neural networks. |
| Outcome: | The proposed model achieves state-of-the-art results on three translation benchmarks with comparable performance to autoregressive and iterative NAT systems. |
Integrating Translation Memories into Non-Autoregressive Machine Translation (2023.eacl-main)
Copied to clipboard
| Challenge: | Non-autoregressive machine translation (NAT) has made great progress, but most studies focus on standard translation tasks. |
| Approach: | They propose to train an edit-based NAT model with a Translation Memory (TM) they propose to modify the data presentation and introduce an extra deletion operation to reduce decoding load. |
| Outcome: | The proposed model performs on par with an autoregressive approach while reducing the decoding load. |
Context-Aware Non-Autoregressive Document-Level Translation with Sentence-Aligned Connectionist Temporal Classification (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies employ autoregressive translation (AT) methods to encode sentences . however, the AT methods struggle with error accumulation when the length of sentences increases. |
| Approach: | They propose a context-aware non-autoregressive framework with the sentence-aligned connectionist temporal classification loss for document-level neural machine translation. |
| Outcome: | The proposed framework achieves 46X speedup on three benchmarks compared to strong baselines. |
Assessing Non-autoregressive Alignment in Neural Machine Translation via Word Reordering (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing non-autoregressive neural machine translation models that implicitly model dependencies are sub-optimal in handling word order errors. |
| Approach: | They propose to learn a non-autoregressive language model that can be combined with Viterbi decoding to achieve better reordering performance. |
| Outcome: | The proposed model outperforms state-of-the-art reordering mechanisms under different word permutation settings with a 2-27 BLEU improvement, suggesting high potential for word alignment in NAT. |
Hybrid-Regressive Paradigm for Accurate and Speed-Robust Neural Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Autoregressive translation (NAT) is less robust in decoding batch size and hardware settings than NAT. |
| Approach: | They propose a two-stage translation prototype that prompts a small number of AT predictions and fills in previously skipped tokens at once. |
| Outcome: | The proposed translation prototype achieves comparable translation quality with AT while having 1.5x faster inference speed regardless of batch size and device. |
Progressive Multi-Granularity Training for Non-Autoregressive Translation (2021.findings-acl)
Copied to clipboard
| Challenge: | Non-autoregressive translation models are weak at learning high-mode knowledge, argues a new study . despite the improved learning difficulty, there are still complicated word orders and structures in the synthetic sentences, making the NAT performance sub-optimal. |
| Approach: | They propose to train non-autoregressive translation models to learn fine-grained lower-mode knowledge . they break down sentence-level examples into three types and increase granularities . |
| Outcome: | The proposed method improves phrase translation accuracy and model reordering ability against strong NAT baselines. |
Non-Autoregressive Neural Machine Translation: A Call for Clarity (2022.emnlp-main)
Copied to clipboard
| Challenge: | Non-autoregressive translation models require a single forward pass to generate the output sequence instead of iteratively producing each predicted token. |
| Approach: | They propose to use a single forward pass to generate the output sequence instead of iteratively producing each predicted token. |
| Outcome: | The proposed models improve translation quality and speed under third-party testing environments. |