Controlling Learned Effects to Reduce Spurious Correlations in Text Classifiers (2023.acl-long)
Copied to clipboard
| Challenge: | toxicity and IMDB review datasets show that pre-trained NLP classifiers learn spurious correlations between input features and label . |
| Approach: | They propose an algorithm to regularize the learnt effect of features on the model’s prediction to the estimated effect of a feature on label. |
| Outcome: | The proposed method minimises spurious correlations and improves minority group accuracy while improving total accuracy compared to standard training. |
Similar Papers
Identifying Spurious Correlations for Robust Text Classification (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Text classifiers often rely on spurious correlations to predict positive reviews . term Spielberg does not cause the review to be positive, so it does not affect the classification accuracy. |
| Approach: | They propose a method to distinguish spurious and genuine correlations in text classification using treatment effect estimators. |
| Outcome: | The proposed method works well even with limited training examples and is possible to transport the word classifier to new domains. |
Fighting Spurious Correlations in Text Classification via a Causal Learning Perspective (2025.naacl-long)
Copied to clipboard
| Challenge: | In text classification tasks, models often rely on spurious correlations for predictions, incorrectly associating irrelevant features with the target labels. |
| Approach: | They propose a Causally Calibrated Robust Classifier which integrates a causal feature selection method based on counterfactual reasoning and an unbiased inverse propensity weighting (IPW) loss function. |
| Outcome: | The proposed method achieves state-of-the-art performance among methods without group labels and can compete with the models that utilize group labels. |
Influence Tuning: Demoting Spurious Correlations via Instance Attribution and Instance-Driven Updates (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to interpret black-box models to learn spurious correlations are not well understood. |
| Approach: | They propose a procedure that leverages model interpretations to update parameters towards a plausible interpretation rather than an interpretation that relies on spurious patterns in data. |
| Outcome: | The proposed procedure outperforms baseline methods that use adversarial training in a controlled setup. |
Understanding and Mitigating Spurious Correlations in Text Classification with Neighborhood Analysis (2024.findings-eacl)
Copied to clipboard
| Challenge: | Recent research has revealed that machine learning models have a tendency to leverage spurious correlations that exist in the training set but may not hold true in general circumstances. |
| Approach: | They propose a metric to detect spurious tokens and a family of regularization methods to mitigate spurious correlations in text classification. |
| Outcome: | The proposed method prevents spurious clusters and significantly improves the robustness of classifiers without auxiliary data. |
Generating Data to Mitigate Spurious Correlations in Natural Language Inference Datasets (2022.acl-long)
Copied to clipboard
| Challenge: | Natural language processing models exploit spurious correlations between features and labels in datasets to perform well only within the distributions they are trained on. |
| Approach: | They propose to generate a debiased version of a dataset and replace it with training data to train a model that is generalised to different task distributions. |
| Outcome: | The proposed method outperforms or performs comparable to state-of-the-art debiasing strategies on a large suite of debiased, out-of distribution, and adversarial test sets. |
Identifying and Mitigating Spurious Correlations for Improving Robustness in NLP Models (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing work identifies task-specific shortcuts via human priors or error analyses, which requires extensive expertise and efforts. |
| Approach: | They propose to automatically identify spurious correlations in NLP models at scale by using existing interpretability methods to extract tokens that significantly affect model’s decision process. |
| Outcome: | The proposed method can identify spurious correlations in NLP models at scale and mitigate these leads to more robust models in multiple applications. |
Are All Spurious Features in Natural Language Alike? An Analysis through a Causal Lens (2022.emnlp-main)
Copied to clipboard
| Challenge: | 'spurious correlations' have been used in NLP to informally denote any undesirable feature-label correlations. |
| Approach: | They formalize this distinction using a causal model and probabilities of necessity and sufficiency, which delineates causal relations between a feature and a label. |
| Outcome: | The proposed model is invariant to the feature, but not sufficient for prediction. |
Stubborn Lexical Bias in Data and Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work has focused on spurious correlations between features and labels in training data . but, we find strong evidence of corresponding bias in the trained models . |
| Approach: | They propose a method to reduce spurious correlations in training data by reweighting it using a large pool of extracted features. |
| Outcome: | The proposed method reduces spurious correlations in training data, but still finds strong evidence of bias in trained models. |
Analyzing Biases to Spurious Correlations in Text Classification Tasks (2022.aacl-short)
Copied to clipboard
| Challenge: | Often these systems exceed human performance, but there is a caveat: standard benchmarks often assume that training and evaluation data are drawn independently and identically from the same underlying distribution. |
| Approach: | They propose to exploit spurious correlations in training data to exploit these correlations . they show that even when only ‘stop’ words are available, it is possible to predict the class significantly better than random. |
| Outcome: | The proposed model can predict class significantly better when only ‘stop’ words are available at the input stage, but can degrade the ability of the system to generalize well to out-of-domain data. |
Increasing Robustness to Spurious Correlations using Forgettable Examples (2021.eacl-main)
Copied to clipboard
| Challenge: | Neural NLP models often exploit spurious correlations to perform tasks. minority examples have been shown to increase the out-of-distribution generalization of pre-trained language models. |
| Approach: | They propose to use example forgetting to find minority examples without prior knowledge of spurious correlations in the dataset. |
| Outcome: | The proposed approach improves out-of-distribution generalization on minorities . it shows that minority examples are more robust on challenging datasets . |