Improving Neural Machine Translation with Conditional Sequence Generative Adversarial Nets (N18-1)
Copied to clipboard
| Challenge: | Experimental results show that the proposed model consistently outperforms the traditional RNNSearch and the newly emerged state-of-the-art Transformer on English-German and Chinese-English translation tasks. |
| Approach: | They propose an approach for applying GANs to NMT by building a conditional sequence generative adversarial net with two adversarials. |
| Outcome: | The proposed model outperforms the existing RNNSearch and Transformer on English-German and Chinese-English translation tasks. |
Similar Papers
Effective Adversarial Regularization for Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Existing (small) perturbations that induce a critical prediction error in machine learning models are often referred to as adversarial examples. |
| Approach: | They propose to use adversarial perturbations to regularize text classification tasks by adding adversarials to a typical NMT model structure. |
| Outcome: | The proposed method significantly improves performance of NMT models, such as LSTM-based and Transformer-based models. |
AdvAug: Robust Adversarial Augmentation for Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Recent work in neural machine translation has led to dramatic improvements in both research and commercial systems. |
| Approach: | They propose a adversarial augmentation method for Neural Machine Translation that minimizes vicinal risk over virtual sentences . they use a novel vicinity distribution for adversarials to describe a smooth interpolated embedding space . |
| Outcome: | The proposed method outperforms the current method on Chinese-English, English-French, and English-German translation benchmarks. |
Crafting Adversarial Examples for Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Effective adversary generation for neural machine translation is crucial for robust systems. |
| Approach: | They propose to leverage round-trip translation technique to build valid metrics for evaluating NMT adversarial attacks. |
| Outcome: | The proposed method could break the state-of-art NMT models with small perturbations. |
An End-to-End Generative Architecture for Paraphrase Generation (D19-1)
Copied to clipboard
| Challenge: | Existing methods for generating paraphrases with linguistic knowledge are often domain specific and hard to scale, or yield inferior results. |
| Approach: | They propose an end-to-end conditional generative architecture for generating paraphrases via adversarial training which does not depend on extra linguistic information. |
| Outcome: | The proposed method outperforms existing models on automatic metrics and human evaluations on four public datasets. |
Best Student Forcing: A Simple Training Mechanism in Adversarial Language Generation (2020.lrec-1)
Copied to clipboard
| Challenge: | Language models trained with Maximum Likelihood Estimation (MLE) have been considered as a mainstream solution in Natural Language Generation (NLG) however, they are reportedly suffering from training instability and mode collapse, and therefore outperform conventional MLE models. |
| Approach: | They propose a method to improve Generative Adversarial Nets (GANs) using best student forcing and discriminators to increase training stability and sample diversity. |
| Outcome: | The proposed techniques outperform MLE models and outperformed existing approaches in terms of sample diversity and training stability. |
Unsupervised Neural Machine Translation with Weight Sharing (P18-1)
Copied to clipboard
| Challenge: | Unsupervised neural machine translation (NMT) is a new approach for machine translation . the model uses only one shared encoder to map pairs of sentences from different languages to a shared-latent space . |
| Approach: | They propose an unsupervised approach which trains the model without labeling data . they propose two independent encoders but share some partial weights to extract high-level representations of input sentences. |
| Outcome: | The proposed approach achieves significant improvements on English-German, English-French and Chinese-to-English translation tasks. |
Deconvolution-Based Global Decoding for Neural Machine Translation (C18-1)
Copied to clipboard
| Challenge: | Existing models for Neural Machine Translation (NMT) use Recurrent Neural Network (RNN) to generate translation word by word following a sequential order. |
| Approach: | They propose a Neural Machine Translation (NMT) model that decodes the sequence with the guidance of its structural prediction of the target-side context. |
| Outcome: | The proposed model is more competitive compared with the state-of-the-art methods and reduces repetition with the instruction from the target-side context for decoding. |
Neural Mask Generator: Learning to Generate Adaptive Word Maskings for Language Model Adaptation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to train language models on diverse text corpora have brought up performance improvements on several natural language understanding (NLU) tasks. |
| Approach: | They propose a method to automatically generate domain- and task-adaptive maskings of a given text for self-supervised pre-training. |
| Outcome: | The proposed framework outperforms rule-based masking strategies on question answering and text classification datasets on which it outperformed rule-driven masking techniques. |
Universal Conditional Masked Language Pre-training for Neural Machine Translation (2022.acl-long)
Copied to clipboard
| Challenge: | Pre-trained sequence-to-sequence models have significantly improved Neural Machine Translation (NMT) this paper demonstrates that pre-training a sequence- to-squence model with a bidirectional decoder can produce notable performance gains for both Autoregressive and Non-autoregressive NMT tasks. |
| Approach: | They propose a conditional masked language model pre-trained on bilingual and monolingual corpora in many languages. |
| Outcome: | The proposed model can achieve significant performance improvements on all scenarios from low- to extremely high-resource languages. |
Pronoun-Targeted Fine-tuning for NMT with Hybrid Losses (2020.emnlp-main)
Copied to clipboard
| Challenge: | Popular machine translation model training uses backtranslation to improve BLEU scores . we use generative-discriminative hybrid losses to fine-tune a trained model . |
| Approach: | They propose a class of conditional generative-discriminative hybrid losses to fine-tune a machine translation model. |
| Outcome: | The proposed model improves on a sentence-level and contextual model without additional data. |