Text Counterfactuals via Latent Optimization and Shapley-Guided Search

Challenge:	Using latent optimization and Shapley values, we generate a set of minimal modifications to the text to change the classifier's prediction.
Approach:	They propose to generate a counterfactual by making minimal modifications to the text to change the model's prediction.
Outcome:	The proposed approach achieves favorable performance compared to white-box and black-box baselines using human and automatic evaluations.

Similar Papers

Overview

Challenge

Approach

Outcome

A Survey on Natural Language Counterfactual Generation (2024.findings-emnlp)

Copied to clipboard

Yongjie Wang, Xiaoqi Qiu, Yu Yue, Xu Guo, Zhiwei Zeng, Yuhong Feng, Zhiqi Shen

Challenge:	Recent advances in NLP are driven by a variety of Large Language Models (LLMs), such as GPT-3 (175B) and PaLM (540B).
Approach:	They propose a taxonomy that categorizes the methods into four groups and summarizes the metrics for evaluating the generation quality.
Outcome:	The proposed taxonomy categorizes the generation methods into four groups and summarizes the metrics for evaluating the quality.

Counterfactuals to Control Latent Disentangled Text Representations for Style Transfer (2021.acl-short)

Copied to clipboard

Sharmila Reddy Nangi, Niyati Chhaya, Sopan Khosla, Nikhil Kaushik, Harshit Nyati

Representation Learning

Challenge:	Existing methods for unsupervised text style transfer focus on transferring a specific attribute, but this technique has never been explored in natural language generation tasks.
Approach:	They propose a counterfactual-based method to modify latent representations by posing a ‘what-if’ scenario.
Outcome:	The proposed method is tested on multiple attribute transfer tasks like Sentiment, Formality and Excitement to support the hypothesis.

Latent-Variable Generative Models for Data-Efficient Text Classification (D19-1)

Copied to clipboard

Xiaoan Ding, Kevin Gimpel

Information Retrieval

Challenge:	Generative classifiers offer potential advantages over discriminative classifications, including data efficiency and zero-shot learning.
Approach:	They introduce discrete latent variables into generative story to improve classifiers' performance . they empirically characterize performance of their models on six text classification datasets .
Outcome:	The proposed model outperforms discriminative and generative classifiers on six text classification datasets.

Exploring the Efficacy of Automatically Generated Counterfactuals for Sentiment Analysis (2021.acl-long)

Copied to clipboard

Linyi Yang, Jiazheng Li, Padraig Cunningham, Yue Zhang, Barry Smyth, Ruihai Dong

Sentiment Analysis

Challenge:	Existing approaches to improve performance of deep neural models are limited by the nature of spurious patterns in the data.
Approach:	They propose to use augmented data to generate spurious patterns in NLP models . they propose to generate counterfactual data for data augmentation and explanation .
Outcome:	The proposed approach improves performance on augmented data and on human-generated data.

NeuroCounterfactuals: Beyond Minimal-Edit Counterfactuals for Richer Data Augmentation (2022.findings-emnlp)

Copied to clipboard

Phillip Howard, Gadi Singer, Vasudev Lal, Yejin Choi, Swabha Swayamdipta

Responsible & Trustworthy NLP

Challenge:	Existing approaches to produce counterfactuals rely on small perturbations via minimal edits, resulting in simplistic changes.
Approach:	They propose a novel approach to produce counterfactuals that allow for larger edits and linguistic diversity while still bearing similarity to the original document.
Outcome:	The proposed approach outperforms existing methods for generalizing natural language models under select settings.

A Practical Method for Generating String Counterfactuals (2025.findings-naacl)

Copied to clipboard

Matan Avitan, Ryan Cotterell, Yoav Goldberg, Shauli Ravfogel

Challenge:	Interventions targeting the representation space of language models (LMs) have emerged as an effective means to influence model behavior.
Approach:	They propose a method to convert representation counterfactuals into string counterf actuals and analyze the linguistic alterations corresponding to the intervention.
Outcome:	The proposed method analyzes linguistic alterations corresponding to a given representation space intervention and interprets features utilized to encode a specific concept.

Copied to clipboard

Jianzhu Bao, Haozhen Zhang, Kuicai Dong, Bozhi Wu, Sarthak Ketanbhai Modi, Zi Pong Lim, Yon Shin Teo, Wenya Wang

Multimodality

Challenge:	Chart understanding is a critical capability for vision-language models, serving as a cornerstone for automated data analysis, document understanding, and scientific research.
Approach:	They propose a chart-efficient training framework to enhance counterfactual sensitivity by code modification and a similarity-based data selection strategy.
Outcome:	The proposed framework achieves superior or comparable performance to strong chart-specific VLMs while using significantly less training data.

AutoCAD: Automatically Generate Counterfactuals for Mitigating Shortcut Learning (2022.findings-emnlp)

Copied to clipboard

Jiaxin Wen, Yeshuang Zhu, Jinchao Zhang, Jie Zhou, Minlie Huang

Challenge:	Existing methods for generating counterfactuals rely on human efforts or task-specific designs.
Approach:	They propose to use a fully automatic and task-agnostic CAD generation framework to generate diverse counterfactuals.
Outcome:	The proposed framework outperforms human-in-the-loop and task-specific CAD methods on multiple out-of-domain and challenge benchmarks.

Retrieval-guided Counterfactual Generation for QA (2022.acl-long)

Copied to clipboard

Bhargavi Paranjape, Matthew Lamm, Ian Tenney

Information Retrieval

Challenge:	Recent work shows that data augmentation using counterfactuals can help ameliorate this weakness.
Approach:	They propose a Retrieve-Generate-Filter technique to generate counterfactuals using QA framework and question generation model trained on original task data.
Outcome:	The proposed method improves performance on out-of-domain and challenging evaluation sets over and above existing methods.

Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models (2021.acl-long)

Copied to clipboard

Tongshuang Wu, Marco Tulio Ribeiro, Jeffrey Heer, Daniel Weld

Explainability & Interpretability in NLP

Challenge:	Existing counterfactual generation methods rely on manual labor to create very few counterf actuals or only instantiate limited types of perturbations such as paraphrases or word substitutions.
Approach:	They propose a general-purpose counterfactual generator that allows for control over perturbation types and locations.
Outcome:	The proposed generator produces diverse sets of realistic counterfactuals that are useful in various applications.