AligNART: Non-autoregressive Neural Machine Translation by Jointly Learning to Estimate Alignment and Translate (2021.emnlp-main)
Copied to clipboard
| Challenge: | Non-autoregressive neural machine translation models suffer from the multi-modality problem . aligNART leverages full alignment information to explicitly reduce the modality of the target distribution . |
| Approach: | They propose an alignment decomposition method which explicitly reduces the modality of the target distribution. |
| Outcome: | The proposed model outperforms previous models that focus on modality reduction on two translation tasks. |
Similar Papers
Assessing Non-autoregressive Alignment in Neural Machine Translation via Word Reordering (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing non-autoregressive neural machine translation models that implicitly model dependencies are sub-optimal in handling word order errors. |
| Approach: | They propose to learn a non-autoregressive language model that can be combined with Viterbi decoding to achieve better reordering performance. |
| Outcome: | The proposed model outperforms state-of-the-art reordering mechanisms under different word permutation settings with a 2-27 BLEU improvement, suggesting high potential for word alignment in NAT. |
Hint-Based Training for Non-Autoregressive Machine Translation (D19-1)
Copied to clipboard
| Challenge: | AutoRegressive Translation models have to generate tokens sequentially during decoding and thus suffer from high inference latency. |
| Approach: | They propose to use hidden states and word alignments to help train NART models. |
| Outcome: | The proposed model improves on the WMT14 En-De and De-En datasets but is faster in inference than the current models. |
Improving Non-autoregressive Neural Machine Translation with Monolingual Data (2020.acl-main)
Copied to clipboard
| Challenge: | Neural machine translation is usually done via knowledge distillation from an autoregressive (AR) model. |
| Approach: | They leverage large monolingual corpora to improve the NAR model's performance by transferring the autoregressive model' s generalization ability while preventing overfitting. |
| Outcome: | The proposed methods on the WMT14 En-De and WMT16 En-Ro news translation tasks show that monolingual data augmentation improves the NAR model to approach the teacher AR model’s performance. |
Non-Autoregressive Neural Machine Translation: A Call for Clarity (2022.emnlp-main)
Copied to clipboard
| Challenge: | Non-autoregressive translation models require a single forward pass to generate the output sequence instead of iteratively producing each predicted token. |
| Approach: | They propose to use a single forward pass to generate the output sequence instead of iteratively producing each predicted token. |
| Outcome: | The proposed models improve translation quality and speed under third-party testing environments. |
Fully Non-autoregressive Neural Machine Translation: Tricks of the Trade (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing non-autoregressive neural machine translation models are slow to learn the dependency between output tokens. |
| Approach: | They propose to use fully non-autoregressive neural machine translation (NAT) to predict tokens with single forward of neural networks. |
| Outcome: | The proposed model achieves state-of-the-art results on three translation benchmarks with comparable performance to autoregressive and iterative NAT systems. |
Enriching Non-Autoregressive Transformer with Syntactic and Semantic Structures for Neural Machine Translation (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing non-autoregressive models have boosted the efficiency of neural machine translation, but their performance is significantly worse than that of autoregressive counterparts. |
| Approach: | They propose to incorporate syntactic and semantic structures among natural languages into a non-autoregressive Transformer for the task of neural machine translation. |
| Outcome: | The proposed model achieves faster speed and keeps translation quality compared with other models. |
Non-Autoregressive Machine Translation with Latent Alignments (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing non-autoregressive machine translation methods are lacking in the field of latent alignments. |
| Approach: | They propose two strong methods for non-autoregressive machine translation that model latent alignments with dynamic programming. |
| Outcome: | The proposed models achieve state-of-the-art on the WMT’14 EnDe task, compared with the autoregressive Transformer baseline. |
End-to-End Non-Autoregressive Neural Machine Translation with Connectionist Temporal Classification (D18-1)
Copied to clipboard
| Challenge: | Autoregressive decoding is the only part of sequence-to-sequence models that prevents massive parallelization at inference time. |
| Approach: | They propose a non-autoregressive architecture based on connectionist temporal classification . they conduct experiments on the WMT English-Romanian and English-German datasets . |
| Outcome: | The proposed model achieves a significant speedup over autoregressive models . the model can be trained end-to-end and maintains translation quality comparable to other models compared to autoregression models based on connectionist temporal classification . |
Learning to Rewrite for Non-Autoregressive Neural Machine Translation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing non-autoregressive neural machine translations have poor inference speed but weak recognition of erroneous translation pieces. |
| Approach: | They propose an architecture to explicitly learn to rewrite the erroneous translation pieces. |
| Outcome: | The proposed architecture can achieve better performance while significantly reducing decoding time. |
Learning to Recover from Multi-Modality Errors for Non-Autoregressive Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing non-autoregressive neural machine translation models suffer from multi-modality problem . despite their autoregressivity, most NMT models suffer with slow decoding speed . |
| Approach: | They propose a semi-autoregressive model which generates a translation as a sequence of segments while each segment is predicted token-by-token. |
| Outcome: | The proposed model can achieve 4 times speedup while maintaining comparable performance. |