Quick Back-Translation for Unsupervised Machine Translation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Unsupervised machine translation models are limited by the run-time of autoregressive inference during back-translation and lack of synthetic data efficiency. |
| Approach: | They propose a two-for-one improvement to Transformer back-translation: Quick Back-Translation (QBT). QBT re-purposes the encoder as a generative model, and uses encoder-generated sequences to train the decoder. |
| Outcome: | Experiments on various WMT benchmarks show that QBT dramatically outperforms standard back-translation only method in terms of training efficiency for comparable translation qualities. |
Similar Papers
Syntactically Supervised Transformers for Faster Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Standard decoders for neural machine translation generate a single token per timestep, which slows inference . a series of controlled experiments demonstrates that SynST decodes sentences 5x faster than the baseline autoregressive Transformer. |
| Approach: | They propose a syntactically supervised Transformer that generates all target tokens in one shot . synST is a variant of the Transformer architecture that autoregressively predicts a chunked parse tree . |
| Outcome: | The proposed method decodes sentences 5x faster than the baseline method on En-De and En-Fr datasets while achieving higher BLEU scores. |
Summarize and Generate to Back-translate: Unsupervised Translation of Programming Languages (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent developments of multilingual pre-trained sequence-to-sequence models for programming languages have been effective for a broad spectrum of downstream software engineering tasks. |
| Approach: | They propose to combine a source-to-target model with a target-tosource model trained in parallel. |
| Outcome: | The proposed approach performs competitively with state-of-the-art methods. |
An Effective Approach to Unsupervised Machine Translation (P19-1)
Copied to clipboard
| Challenge: | a recent research line has managed to train both unsupervised and unsupervised machine translation systems using monolingual corpora only. |
| Approach: | They propose to use monolingual corpora to train both unsupervised and unsupervised machine translation systems. |
| Outcome: | The proposed system achieves 22.5 BLEU points in English-to-German WMT 2014, 5.5 points more than the previous best unsupervised system, and 0.5 points more in the (supervised) shared task winner back in 2014. |
TranSFormer: Slow-Fast Transformer for Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Prior work has focused on treating subwords as basic units in developing such systems. |
| Approach: | They propose a slow-fast two-stream learning model that uses a “slow” branch to deal with subword sequences and a "fast" branch to cope with longer character sequences. |
| Outcome: | The proposed model shows consistent BLEU improvements (larger than 1 BLUE point) on several machine translation benchmarks. |
Tagged Back-translation Revisited: Why Does It Really Work? (2020.acl-main)
Copied to clipboard
| Challenge: | In this paper, we show that neural machine translation systems trained on large back-translated data overfit some of the characteristics of machine-transcribed texts. |
| Approach: | They propose to add a tag to back-translations to help distinguish back-translated data from original parallel training data. |
| Outcome: | The proposed tag helps the system distinguish back-translated data from original parallel training data and is as effective as a tag in high-resource training. |
Semi-Autoregressive Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Existing approaches to neural machine translation are typically autoregressive but suffer from low parallelizability and thus slow at decoding long sequences. |
| Approach: | They propose a semi-autoregressive Transformer model for fast sequence generation that keeps the autoregressive property in global but relieves in local . |
| Outcome: | The proposed model achieves 5.58 speedup while maintaining 88% translation quality, significantly better than previous non-autoregressive methods. |
Translating Translationese: A Two-Step Approach to Unsupervised Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Using a dictionary, given a rough, target language natives can uncover the latent, fully-fluent rendering of the translation. |
| Approach: | They propose a method that breaks translation into two steps by generating a dictionary and then ‘translating’ the resulting pseudo-translation into a fully fluent translation. |
| Outcome: | The proposed method 'gets better translation results on high-resource languages than previously published unsupervised MT studies' |
On-the-Fly Fusion of Large Language Models and Machine Translation (2024.findings-naacl)
Copied to clipboard
| Challenge: | a weaker-at-translation LLM can improve translations of a NMT model, compared to a strong dedicated model. |
| Approach: | They propose to ensemble a neural machine translation model with a large language model, prompted on the same task and input. |
| Outcome: | The proposed method can be combined with various techniques from LLM prompting, such as in context learning and translation context. |
A Retrieve-and-Rewrite Initialization Method for Unsupervised Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Recent work shows successful methods for unsupervised machine translation (UMT) initialization stage is important since bad initialization may wrongly squeeze the search space and too much noise may hurt the final performance. |
| Approach: | They propose a retrieval and rewriting based method to better initialize unsupervised translation models. |
| Outcome: | The proposed method improves translation performance by over 4 BLEU scores. |
Non-autoregressive Streaming Transformer for Simultaneous Translation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Simultaneous machine translation models are trained to strike a balance between latency and translation quality. |
| Approach: | They propose a non-autoregressive streaming Transformer which generates blank tokens and decodes repetitive tokens to adjust its READ/WRITE strategy flexibly. |
| Outcome: | The proposed model outperforms previous strong autoregressive models on various benchmarks on siMT. |