Manifold Adversarial Augmentation for Neural Machine Translation (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent studies show that NMT models can drop significantly when small perturbations are added to input sentences. |
| Approach: | They propose a data augmentation approach to sample sentences from the vicinity distributions in higher-level representations. |
| Outcome: | The proposed method improves translation accuracy on training samples from higher-level representations. |
Similar Papers
AdvAug: Robust Adversarial Augmentation for Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Recent work in neural machine translation has led to dramatic improvements in both research and commercial systems. |
| Approach: | They propose a adversarial augmentation method for Neural Machine Translation that minimizes vicinal risk over virtual sentences . they use a novel vicinity distribution for adversarials to describe a smooth interpolated embedding space . |
| Outcome: | The proposed method outperforms the current method on Chinese-English, English-French, and English-German translation benchmarks. |
Counterfactual Data Augmentation for Neural Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Neural machine translation models often rely on large-scale parallel corpora for training, exhibiting degraded performance on low-resource languages. |
| Approach: | They propose a method that interprets language models and phrasal alignment causally and generates augmented parallel translation corpora by sampling new source phrases from a masked language model. |
| Outcome: | The proposed method improves translation, backtranslation and translation robustness on IWSLT’15 English Vietnamese, WMT’17 English - German, and WMT'18 English – Turkish. |
End-to-end Adversarial Sample Generation for Data Augmentation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for generating adversarial samples have deceived many neural inference models, such as text classification and machine translation. |
| Approach: | They propose an adversarial sample generator that consists of a conditioned paraphrasing model and a condition generator and introduce a pretrained discriminator to help the adversarial sample generator adapt to the data characteristics. |
| Outcome: | The proposed approach improves the performance of the trained model on several tasks and is robust for various attacking techniques. |
Virtual Augmentation Supported Contrastive Learning of Sentence Representations (2022.findings-acl)
Copied to clipboard
| Challenge: | Despite profound successes, contrastive representation learning relies on carefully designed data augmentations using domain-specific knowledge. |
| Approach: | They propose a virtual augmentation supported Contrastive Learning of sentence representations . they approximate the neighborhood of an instance via its K-nearest in-batch neighbors . |
| Outcome: | The proposed model outperforms existing methods on a wide range of downstream tasks. |
Sentence Concatenation Approach to Data Augmentation for Neural Machine Translation (2021.naacl-srw)
Copied to clipboard
| Challenge: | Neural machine translation is known to show poor performance at long sentence translations . however, when the sentence length exceeds a certain value, the quality of NMT becomes inferior to that of statistical machine translation. |
| Approach: | They propose a method that uses given parallel corpora as train data to generate long sentences by concatenating two sentences at random. |
| Outcome: | The proposed method improves translation quality more when combined with back-translation. |
Data Augmentation with Adversarial Training for Cross-Lingual NLI (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to train cross-lingual models with labeled data are subpar, resulting in subpar results. |
| Approach: | They propose a data augmentation strategy that enriches data to reflect more diversity in a semantically faithful way and leverages adversarial training regimens to achieve greater robustness. |
| Outcome: | The proposed approach improves cross-lingual inference by leveraging the data to reflect more diversity in a semantically faithful way. |
Rethinking Data Augmentation for Low-Resource Neural Machine Translation: A Multi-Task Learning Approach (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to generating additional parallel sentences are aimed at expanding the support of the empirical data distribution by generating new sentence pairs that contain infrequent words. |
| Approach: | They propose to use data augmentation techniques to generate additional parallel sentences by reversing the order of the target sentence to produce unfluent target sentences. |
| Outcome: | The proposed approach improves on six low-resource translation tasks and the baseline and over DA methods. |
Virtual Data Augmentation: A Robust and General Framework for Fine-tuning Pre-trained Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have shown that powerful pre-trained language models can be fooled by small perturbations or intentional attacks. |
| Approach: | They propose a framework for fine-tuning PLMs using a masked language model and Gaussian noise to augment semantically relevant examples with sufficient diversity. |
| Outcome: | The proposed framework improves the robustness of pre-trained language models and alleviates performance degradation under adversarial attacks. |
Adversarial Augmentation Policy Search for Domain and Cross-Lingual Generalization in Reading Comprehension (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Reading comprehension models often overfit to nuances of training datasets and fail at adversarial evaluation. |
| Approach: | They propose a method that introduces multiple points of confusion within the context and shows dependence on insertion location of the distractor. |
| Outcome: | The proposed methods improve robustness against adversarial evaluation but weak generalization to the source domain and new domains and languages. |
Boosting Neural Machine Translation with Similar Translations (2020.acl-main)
Copied to clipboard
| Challenge: | Statistical Machine Translation and fuzzy matching are completely different in their finality. |
| Approach: | They propose to use fuzzy matching to train neural machine translation to make use of similar translations, in a similar way a human translator employs fuzzy matches. |
| Outcome: | The proposed methods improve translation accuracy and fine-tuned model for unseen translation pairs. |