AutoCAD: Automatically Generate Counterfactuals for Mitigating Shortcut Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for generating counterfactuals rely on human efforts or task-specific designs. |
| Approach: | They propose to use a fully automatic and task-agnostic CAD generation framework to generate diverse counterfactuals. |
| Outcome: | The proposed framework outperforms human-in-the-loop and task-specific CAD methods on multiple out-of-domain and challenge benchmarks. |
Similar Papers
Exploring the Efficacy of Automatically Generated Counterfactuals for Sentiment Analysis (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to improve performance of deep neural models are limited by the nature of spurious patterns in the data. |
| Approach: | They propose to use augmented data to generate spurious patterns in NLP models . they propose to generate counterfactual data for data augmentation and explanation . |
| Outcome: | The proposed approach improves performance on augmented data and on human-generated data. |
People Make Better Edits: Measuring the Efficacy of LLM-Generated Counterfactually Augmented Data for Harmful Language Detection (2023.emnlp-main)
Copied to clipboard
| Challenge: | Past work has shown that counterfactually augmented data (CADs) can improve models' performance on out-of-domain tests. |
| Approach: | They use Polyjuice, ChatGPT, and Flan-T5 to automatically generate CADs . they find that CAD generates a model that flips the original label with minimal changes . |
| Outcome: | The proposed model improves model robustness on out-of-domain test sets and individual data points. |
PairCFR: Enhancing Model Training on Paired Counterfactually Augmented Data through Contrastive Learning (2024.acl-long)
Copied to clipboard
| Challenge: | Recent research shows that training with CAD may lead models to overly focus on modified features while ignoring other important contextual information. |
| Approach: | They propose to use contrastive learning to promote global feature alignment and learning counterfactual clues to improve model performance. |
| Outcome: | The proposed method outperforms the state-of-the-art on out-of distribution (OOD) datasets. |
How Does Counterfactually Augmented Data Impact Models for Social Computing Constructs? (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have shown that models trained on CAD can learn cues in the dataset which are spuriously correlated with the construct. |
| Approach: | They focus on sentiment, sexism, and hate speech as social constructs to investigate their effects on model performance. |
| Outcome: | The proposed model generalizes better on out-of-domain datasets while relying less on spurious features. |
NeuroCounterfactuals: Beyond Minimal-Edit Counterfactuals for Richer Data Augmentation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to produce counterfactuals rely on small perturbations via minimal edits, resulting in simplistic changes. |
| Approach: | They propose a novel approach to produce counterfactuals that allow for larger edits and linguistic diversity while still bearing similarity to the original document. |
| Outcome: | The proposed approach outperforms existing methods for generalizing natural language models under select settings. |
Improving Classifier Robustness through Active Generative Counterfactual Data Augmentation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for finding meaningful counterfactuals rely on human annotation or implicit label invariance . a small amount of human-annotated counterf actual data can generate a robust dataset with learned labels. |
| Approach: | They propose a framework that generates counterfactuals by actively sampling from regions of uncertainty and automatically labeling them with a learned auxiliary classifier. |
| Outcome: | The proposed framework generates a large number of diverse counterfactuals and labels them with a learned classifier. |
An Investigation of the (In)effectiveness of Counterfactually Augmented Data (2022.acl-long)
Copied to clipboard
| Challenge: | Pretrained language models tend to rely on spurious correlations and generalize poorly to out-of-distribution (OOD) data. |
| Approach: | They propose to use counterfactually-augmented data (CAD) to identify robust features that are invariant under distribution shift to train models for OOD generalization. |
| Outcome: | The proposed model can learn robust features that are invariant under distribution shifts, but lacks spurious correlations, and may exacerbate existing correlations. |
FitCF: A Framework for Automatic Feature Importance-guided Counterfactual Example Generation (2025.findings-acl)
Copied to clipboard
Qianli Wang, Nils Feldhus, Simon Ostermann, Luis Felipe Villa-Arenas, Sebastian Möller, Vera Schmitt
| Challenge: | Existing frameworks for counterfactual examples are lacking for many tasks. |
| Approach: | They propose a faithful approach for leveraging important words from feature attribution methods to generate counterfactual examples in a zero-shot setting. |
| Outcome: | The proposed framework outperforms state-of-the-art frameworks on many tasks. |
Counterfactually Augmented Data and Unintended Bias: The Case of Sexism and Hate Speech Detection (2022.naacl-main)
Copied to clipboard
| Challenge: | sexism and hate speech detection models may be over-relying on core features . construct-driven CAD may induce models to ignore context in which core features are used . |
| Approach: | They propose to use construct-driven and construct-agnostic CAD to reduce model bias . sexism and hate speech detection models are trained on counterfactually augmented data . |
| Outcome: | Using a diverse set of CAD—construct-driven and construct-agnostic—reduces unintended bias. |
DISCO: Distilling Counterfactuals with Large Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | high-quality counterfactual data is scarce for most tasks and not easily generated at scale. |
| Approach: | They propose a method for automatically generating high-quality counterfactual data at scale . they use a large general language model to generate phrasal perturbations and filter them . |
| Outcome: | The proposed method is task-agnostic and can be applied to the task of natural language inference. |