| Challenge: | Recent advances in automatic data augmentation have focused on computer vision tasks where it is easy to apply imperceptible perturbations without changing an image’s semantic meaning. |
| Approach: | They adapt AutoAugment to automatically discover effective perturbation policies for natural language processing (NLP) tasks such as dialogue generation. |
| Outcome: | The proposed algorithm reduces data-level model bias by using a controller trained on the target task. |
Similar Papers
Text AutoAugment: Learning Compositional Augmentation Policy for Text Classification (2021.emnlp-main)
Copied to clipboard
| Challenge: | Data augmentation aims to alleviate the overfitting issue in low-resource or class-imbalanced situations. |
| Approach: | They propose a framework called Text AutoAugment to enhance training samples . they use a Bayesian optimization algorithm to search for the best policy . |
| Outcome: | The proposed framework outperforms baseline methods on six benchmark datasets. |
AutoAugment Is What You Need: Enhancing Rule-based Augmentation Methods in Low-resource Regimes (2024.eacl-srw)
Copied to clipboard
| Challenge: | Existing methods for text data augmentation suffer from potential semantic damage due to the discrete nature of sentences. |
| Approach: | They propose to adapt AutoAugment to solve this problem by using softEDA to increase text data. |
| Outcome: | The proposed method can boost existing augmentation methods and enhance cutting-edge pretrained language models. |
Mitigating Dataset Artifacts in Natural Language Inference Through Automatic Contextual Data Augmentation and Learning Optimization (2022.lrec-1)
Copied to clipboard
| Challenge: | In recent years, natural language inference has been an emerging research area . a new data augmentation technique is used to augment pre-trained language models . |
| Approach: | They propose to combine automatic contextual data augmentation with a learning procedure for natural language inference. |
| Outcome: | The proposed method outperforms baseline pre-trained language models on benchmark datasets and adversarial examples. |
Simple Data Augmentation with the Mask Token Improves Domain Adaptation for Dialog Act Tagging (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on DA tagging focus on human-human social conversations, which is less applicable for task-oriented setting. |
| Approach: | They propose a controllable mechanism that augments text input by leveraging the pre-trained Mask token from BERT model. |
| Outcome: | The proposed mechanism augments text input by leveraging the pre-trained Mask token from BERT model. |
Adversarial Augmentation Policy Search for Domain and Cross-Lingual Generalization in Reading Comprehension (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Reading comprehension models often overfit to nuances of training datasets and fail at adversarial evaluation. |
| Approach: | They propose a method that introduces multiple points of confusion within the context and shows dependence on insertion location of the distractor. |
| Outcome: | The proposed methods improve robustness against adversarial evaluation but weak generalization to the source domain and new domains and languages. |
Self-training Improves Pre-training for Natural Language Understanding (2021.naacl-main)
Copied to clipboard
Jingfei Du, Edouard Grave, Beliz Gunel, Vishrav Chaudhary, Onur Celebi, Michael Auli, Veselin Stoyanov, Alexis Conneau
| Challenge: | Unsupervised pretraining has led to improvements in natural language understanding . a data augmentation method can be used to generate labels for unlabeled examples . |
| Approach: | They propose a semi-supervised method which uses unlabeled data to retrieve sentences from a database of billions of unlabed sentences crawled from the web. |
| Outcome: | The proposed method improves on standard text classification benchmarks by 2.6% and knowledge distillation by few shots. |
Virtual Data Augmentation: A Robust and General Framework for Fine-tuning Pre-trained Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have shown that powerful pre-trained language models can be fooled by small perturbations or intentional attacks. |
| Approach: | They propose a framework for fine-tuning PLMs using a masked language model and Gaussian noise to augment semantically relevant examples with sufficient diversity. |
| Outcome: | The proposed framework improves the robustness of pre-trained language models and alleviates performance degradation under adversarial attacks. |
GDA: Grammar-based Data Augmentation for Text Classification using Slot Information (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies suggest data augmentation approaches to resolve the low-resource problem in natural language processing tasks. |
| Approach: | They propose to use slot information to augment sentences using a set of injective relations between a sentence’s semantics and its syntactical structure to augment the dataset. |
| Outcome: | The proposed approach outperforms all other data augmentation methods by 19.38%. |
Good-Enough Compositional Data Augmentation (2020.acl-main)
Copied to clipboard
| Challenge: | a proposed data augmentation protocol provides a compositional inductive bias in conditional and unconditional sequence models. |
| Approach: | They propose a data augmentation protocol that provides a compositional inductive bias in conditional and unconditional sequence models by replacing discontinuous fragments with other fragments that appear in at least one similar environment. |
| Outcome: | The proposed protocol reduces error rate by 87% on diagnostic tasks and 16% on semantic parsing tasks. |
Generative Data Augmentation for Commonsense Reasoning (2020.findings-emnlp)
Copied to clipboard
Yiben Yang, Chaitanya Malaviya, Jared Fernandez, Swabha Swayamdipta, Ronan Le Bras, Ji-Ping Wang, Chandra Bhagavatula, Yejin Choi, Doug Downey
| Challenge: | Recent advances in commonsense reasoning depend on large-scale human-authored training data. |
| Approach: | They propose a generative data augmentation technique that augments human-authored training data by using pretrained language models. |
| Outcome: | The proposed technique outperforms existing methods on commonsense reasoning benchmarks and enhances out-of-distribution generalization. |