Are All Spurious Features in Natural Language Alike? An Analysis through a Causal Lens (2022.emnlp-main)
Copied to clipboard
| Challenge: | 'spurious correlations' have been used in NLP to informally denote any undesirable feature-label correlations. |
| Approach: | They formalize this distinction using a causal model and probabilities of necessity and sufficiency, which delineates causal relations between a feature and a label. |
| Outcome: | The proposed model is invariant to the feature, but not sufficient for prediction. |
Similar Papers
Informativeness and Invariance: Two Perspectives on Spurious Correlations in Natural Language (2022.naacl-main)
Copied to clipboard
| Challenge: | Spurious correlations are a threat to the trustworthiness of natural language processing systems. |
| Approach: | They propose a definition of spurious correlations in terms of conditional probabilities and a generalized definition of the term . they propose UIs that allow individual input features to be independent of labels. |
| Outcome: | The proposed definition can be generalized from uniformity to independence without affecting the claims of the paper. |
Stubborn Lexical Bias in Data and Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work has focused on spurious correlations between features and labels in training data . but, we find strong evidence of corresponding bias in the trained models . |
| Approach: | They propose a method to reduce spurious correlations in training data by reweighting it using a large pool of extracted features. |
| Outcome: | The proposed method reduces spurious correlations in training data, but still finds strong evidence of bias in trained models. |
Competency Problems: On Finding and Removing Artifacts in Language Data (2021.emnlp-main)
Copied to clipboard
Matt Gardner, William Merrill, Jesse Dodge, Matthew Peters, Alexis Ross, Sameer Singh, Noah A. Smith
| Challenge: | Recent work in NLP has documented dataset artifacts, bias, and spurious correlations . how to tell which features have spurious instead of legitimate correlations is typically left unspecified . |
| Approach: | They propose a class of competency problems to formalize this notion into a classification . they show that realistic datasets will increasingly deviate from competency problems . |
| Outcome: | The proposed model can be used to show that models are inappropriately affected by these less extreme biases. |
Mitigating Spurious Correlations via Counterfactual Contrastive Learning (2025.findings-emnlp)
Copied to clipboard
Fengxiang Cheng, Chuan Zhou, Xiang Li, Alina Leidinger, Haoxuan Li, Mingming Gong, Fenrong Liu, Robert Van Rooij
| Challenge: | Existing methods to distinguish causally related words from spurious correlations are limited by the number of causally correlated words in a sentence. |
| Approach: | They propose to use probabilistic probability of necessity and probability of sufficiency to identify causal relationships rather than spurious correlations between words and class labels. |
| Outcome: | The proposed method is based on a contrastive learning approach name CPNS and is validated on public datasets. |
Identifying Spurious Correlations for Robust Text Classification (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Text classifiers often rely on spurious correlations to predict positive reviews . term Spielberg does not cause the review to be positive, so it does not affect the classification accuracy. |
| Approach: | They propose a method to distinguish spurious and genuine correlations in text classification using treatment effect estimators. |
| Outcome: | The proposed method works well even with limited training examples and is possible to transport the word classifier to new domains. |
Generating Data to Mitigate Spurious Correlations in Natural Language Inference Datasets (2022.acl-long)
Copied to clipboard
| Challenge: | Natural language processing models exploit spurious correlations between features and labels in datasets to perform well only within the distributions they are trained on. |
| Approach: | They propose to generate a debiased version of a dataset and replace it with training data to train a model that is generalised to different task distributions. |
| Outcome: | The proposed method outperforms or performs comparable to state-of-the-art debiasing strategies on a large suite of debiased, out-of distribution, and adversarial test sets. |
Identifying and Mitigating Spurious Correlations for Improving Robustness in NLP Models (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing work identifies task-specific shortcuts via human priors or error analyses, which requires extensive expertise and efforts. |
| Approach: | They propose to automatically identify spurious correlations in NLP models at scale by using existing interpretability methods to extract tokens that significantly affect model’s decision process. |
| Outcome: | The proposed method can identify spurious correlations in NLP models at scale and mitigate these leads to more robust models in multiple applications. |
Decorrelate Irrelevant, Purify Relevant: Overcome Textual Spurious Correlations from a Feature Perspective (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods to debiase samples with biased features obstructs the model in learning from non-biased parts of the samples. |
| Approach: | They propose to eliminate spurious correlations in a fine-grained manner from a feature space perspective by using Random Fourier Features and weighted re-sampling to decorrelate dependencies between features. |
| Outcome: | The proposed method eliminates spurious correlations in a fine-grained manner from a feature space perspective. |
Which Spurious Correlations Impact Reasoning in NLI Models? A Visual Interactive Diagnosis through Data-Constrained Counterfactuals (2023.acl-demo)
Copied to clipboard
| Challenge: | a spurious correlation exists when a feature correlates with the target label while there is no causal relationship between the feature and the label. |
| Approach: | They propose a dashboard that allows users to generate diverse and challenging examples by drawing inspiration from GPT-3 suggestions. |
| Outcome: | The proposed dashboard enables users to generate diverse and challenging examples by drawing inspiration from GPT-3 suggestions and make refinements based on the feedback. |
Controlling Learned Effects to Reduce Spurious Correlations in Text Classifiers (2023.acl-long)
Copied to clipboard
| Challenge: | toxicity and IMDB review datasets show that pre-trained NLP classifiers learn spurious correlations between input features and label . |
| Approach: | They propose an algorithm to regularize the learnt effect of features on the model’s prediction to the estimated effect of a feature on label. |
| Outcome: | The proposed method minimises spurious correlations and improves minority group accuracy while improving total accuracy compared to standard training. |