Consistency Training with Virtual Adversarial Discrete Perturbation (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for regularizing a model are agnostic to the training model and may not be effective for perturbed inputs. |
| Approach: | They propose an augmentation method of adding a discrete noise that would incur the highest divergence between predictions by replacing tokens while keeping original semantics. |
| Outcome: | The proposed method outperforms baselines on semi-supervised text classification tasks and a robustness benchmark. |
Similar Papers
Achieving Model Robustness through Discrete Adversarial Training (2021.emnlp-main)
Copied to clipboard
| Challenge: | Discrete adversarial attacks are symbolic perturbations to a language input that preserve the output label but lead to predicting error. |
| Approach: | They propose a discrete adversarial attack based on best-first search and random sampling attacks that are not based upon expensive search procedures. |
| Outcome: | The proposed attack outperforms offline augmentation and speedups on three datasets. |
Generative Adversarial Training with Perturbed Token Detection for Model Robustness (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing adversarial training methods use discrete tokens to deceive models . current approaches use embeddings, whereas actual text-based training uses discrete text tokens. |
| Approach: | They propose a framework that integrates gradient-based learning, adversarial example generation and perturbed token detection to enhance adversariarial robustness. |
| Outcome: | The proposed framework surpasses the state-of-the-art results of ChatGPT by 10% in average accuracy. |
Virtual Data Augmentation: A Robust and General Framework for Fine-tuning Pre-trained Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have shown that powerful pre-trained language models can be fooled by small perturbations or intentional attacks. |
| Approach: | They propose a framework for fine-tuning PLMs using a masked language model and Gaussian noise to augment semantically relevant examples with sufficient diversity. |
| Outcome: | The proposed framework improves the robustness of pre-trained language models and alleviates performance degradation under adversarial attacks. |
Gradient-Based Adversarial Factual Consistency Evaluation for Abstractive Summarization (2021.emnlp-main)
Copied to clipboard
| Challenge: | Abstractive summarization models often produce inconsistent statements or false facts. |
| Approach: | They propose an efficient weak-supervised adversarial data augmentation approach to generate factual consistency datasets by backpropagating gradients on token embeddings. |
| Outcome: | The proposed model can make interpretable factual errors tracing on public datasets and is cost-effective. |
Unsupervised Paraphrasing Consistency Training for Low Resource Named Entity Recognition (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods augment input sequence with token replacement, assuming annotations on the replaced positions are unchanged. |
| Approach: | They propose to use paraphrasing to enhance unsupervised consistency training by replacing tokens with augmented data. |
| Outcome: | The proposed method is especially effective when annotations are limited. |
Balanced Adversarial Training: Balancing Tradeoffs between Fickleness and Obstinacy in NLP Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Traditional adversarial examples involve finding a small perturbation that does not change an input’s true label but confuses the classifier into outputting a different prediction. |
| Approach: | They propose to use contrastive learning to increase model robustness against fickle adversarial examples by reducing the vulnerability of adversarials to fickle ones. |
| Outcome: | The proposed method improves model robustness against fickle and obstinate adversarial examples. |
Revisiting Adversarial Autoencoder for Unsupervised Word Translation with Cycle Consistency and Improved Training (N19-1)
Copied to clipboard
| Challenge: | Recent work has shown superior performance for non-adversarial methods in more challenging language pairs. |
| Approach: | They propose to use adversarial autoencoder to map monolingual embeddings to a shared space and to put the target encoders as an adversary against the corresponding discriminator. |
| Outcome: | The proposed method is more robust and achieves better performance than previously proposed adversarial and non-adversarial methods. |
Impact of Adversarial Training on Robustness and Generalizability of Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Adversarial training is widely acknowledged as the most effective defense against adversarial attacks, but achieving both robustness and generalization requires a trade-off. |
| Approach: | They propose to compare pre-training data augmentation and training time input perturbations with embedding space perturbations to find out whether they improve generalization. |
| Outcome: | The proposed methods improve generalization and robustness of the trained models. |
READ: Improving Relation Extraction from an ADversarial Perspective (2024.findings-naacl)
Copied to clipboard
| Challenge: | Recent work in relation extraction (RE) has high generalization capability, but adversarial training methods rely on entities. |
| Approach: | They propose an adversarial training method specifically designed for relation extraction that introduces sequence- and token-level perturbations to the sample and uses a separate perturbation vocabulary to improve the search for entity and context perturbations. |
| Outcome: | The proposed method significantly improves accuracy and robustness in low-resource scenarios. |
Learning from Perturbations: Diverse and Informative Dialogue Generation with Inverse Adversarial Training (2021.acl-long)
Copied to clipboard
| Challenge: | Inverse Adversarial Training (IAT) algorithm for training neural dialogue systems to avoid generic responses and model dialogue history better. |
| Approach: | They propose an algorithm that encourages the model to be sensitive to perturbations in dialogue history and learn from perturbations. |
| Outcome: | The proposed approach can model dialogue history better and generate more diverse responses on two benchmark datasets. |