CREST: A Joint Framework for Rationalization and Counterfactual Text Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for analyzing and training NLP models have not been integrated to combine their complementary advantages. |
| Approach: | They introduce a framework for selective rationalization and counterfactual text generation that leverages CREST to regularize selective rationales and a loss function that regularizes selective rationals. |
| Outcome: | The proposed framework generates valid counterfactuals that are more natural than those produced by previous methods and can be used for data augmentation at scale. |
Similar Papers
Measuring Association Between Labels and Free-Text Rationales (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for extractive rationales do not work as well on reasoning tasks requiring free-text rationale. |
| Approach: | They propose to use pipelines to extract rationales from input words and to use them to explain reasoning tasks. |
| Outcome: | The proposed models exhibit desirable properties for explaining commonsense question-answering and natural language inference, indicating their potential for producing faithful free-text rationales. |
EXPLAIN, EDIT, GENERATE: Rationale-Sensitive Counterfactual Data Augmentation for Multi-hop Fact Verification (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to augment training data with counterfactuals fail to handle multi-hop fact verification due to their incapability to preserve complex logical relationships. |
| Approach: | They propose to augment training data with counterfactuals that alter causal features of the original data by preserving logical relationships. |
| Outcome: | The proposed method outperforms the baselines and can generate linguistically diverse counterfactuals without disrupting their logical relationships. |
Counterfactuals of Counterfactuals: a back-translation-inspired approach to analyse counterfactual editors (2023.findings-acl)
Copied to clipboard
George Filandrianos, Edmund Dervakos, Orfeas Menis Mastromichalakis, Chrysoula Zerva, Giorgos Stamou
| Challenge: | Existing explanations for classifiers are counterfactual or contrastive . lack of universal ground truth for counterf actual edits hinders their evaluation . |
| Approach: | They propose a back translation-inspired evaluation methodology that utilises earlier outputs of the explainer as ground truth proxies to investigate the consistency of explainers. |
| Outcome: | The proposed method can provide valuable insights into the behaviour of predictor and explainer models and infer patterns that would otherwise be obscured. |
Can Rationalization Improve Robustness? (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing models that generate rationales before making predictions can ignore noise or adversarially added text by simply masking it out of the generated rationale. |
| Approach: | They propose to use a 'rationalizethen-predict' framework to generate subsets of input to generate rationales and then make predictions using them. |
| Outcome: | The proposed models improve robustness over AddText attacks while struggling in certain scenarios. |
NeuroCounterfactuals: Beyond Minimal-Edit Counterfactuals for Richer Data Augmentation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to produce counterfactuals rely on small perturbations via minimal edits, resulting in simplistic changes. |
| Approach: | They propose a novel approach to produce counterfactuals that allow for larger edits and linguistic diversity while still bearing similarity to the original document. |
| Outcome: | The proposed approach outperforms existing methods for generalizing natural language models under select settings. |
SPECTRA: Sparse Structured Text Rationalization (2021.emnlp-main)
Copied to clipboard
| Challenge: | Sparse attention mechanisms are a deterministic alternative, but they lack a way to regularize rationale extraction. |
| Approach: | They propose a framework for deterministic extraction of structured explanations via constrained inference on a factor graph, forming a differentiable layer. |
| Outcome: | The proposed framework outperforms previous studies on performance and plausibility of extracted rationales. |
Exploring the Trade-off Between Model Performance and Explanation Plausibility of Text Classifiers Using Human Rationales (2024.findings-naacl)
Copied to clipboard
| Challenge: | Saliency post-hoc explainability methods are important tools for understanding complex NLP models, but they may not align with human intuition, making the explanations not plausible. |
| Approach: | They propose a method for incorporating rationales into text classification models by augmenting the standard cross-entropy loss with a novel loss function inspired by contrastive learning. |
| Outcome: | The proposed approach enhances the plausibility of post-hoc explanations while preserving their faithfulness. |
A Survey on Natural Language Counterfactual Generation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in NLP are driven by a variety of Large Language Models (LLMs), such as GPT-3 (175B) and PaLM (540B). |
| Approach: | They propose a taxonomy that categorizes the methods into four groups and summarizes the metrics for evaluating the generation quality. |
| Outcome: | The proposed taxonomy categorizes the generation methods into four groups and summarizes the metrics for evaluating the quality. |
Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models (2021.acl-long)
Copied to clipboard
| Challenge: | Existing counterfactual generation methods rely on manual labor to create very few counterf actuals or only instantiate limited types of perturbations such as paraphrases or word substitutions. |
| Approach: | They propose a general-purpose counterfactual generator that allows for control over perturbation types and locations. |
| Outcome: | The proposed generator produces diverse sets of realistic counterfactuals that are useful in various applications. |
Self-Training Meets Consistency: Improving LLMs’ Reasoning with Consistency-Driven Rationale Evaluation (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing approaches labeled rationales that produce correct answers as appropriate for training but one measure risks misjudging rationale quality, leading models to learn flawed reasoning patterns. |
| Approach: | They propose a framework that evaluates rationales through follow-up questions and leverages this evaluation to guide its training. |
| Outcome: | The proposed framework improves robustness and correctness of rationales and reasoning abilities compared to previous self-training approaches. |