Increasing Robustness to Spurious Correlations using Forgettable Examples (2021.eacl-main)
Copied to clipboard
| Challenge: | Neural NLP models often exploit spurious correlations to perform tasks. minority examples have been shown to increase the out-of-distribution generalization of pre-trained language models. |
| Approach: | They propose to use example forgetting to find minority examples without prior knowledge of spurious correlations in the dataset. |
| Outcome: | The proposed approach improves out-of-distribution generalization on minorities . it shows that minority examples are more robust on challenging datasets . |
Similar Papers
An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models (2020.tacl-1)
Copied to clipboard
| Challenge: | Recent work shows that pre-trained language models perform poorly on challenging datasets where spurious correlations do not hold. |
| Approach: | They propose to use multi-task learning to improve generalization from minority examples . they propose to combine MTL with auxiliary tasks to improve performance . |
| Outcome: | The proposed model generalizes from minority examples without hurting in-distribution performance. |
Stubborn Lexical Bias in Data and Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work has focused on spurious correlations between features and labels in training data . but, we find strong evidence of corresponding bias in the trained models . |
| Approach: | They propose a method to reduce spurious correlations in training data by reweighting it using a large pool of extracted features. |
| Outcome: | The proposed method reduces spurious correlations in training data, but still finds strong evidence of bias in trained models. |
Controlling Learned Effects to Reduce Spurious Correlations in Text Classifiers (2023.acl-long)
Copied to clipboard
| Challenge: | toxicity and IMDB review datasets show that pre-trained NLP classifiers learn spurious correlations between input features and label . |
| Approach: | They propose an algorithm to regularize the learnt effect of features on the model’s prediction to the estimated effect of a feature on label. |
| Outcome: | The proposed method minimises spurious correlations and improves minority group accuracy while improving total accuracy compared to standard training. |
Generating Data to Mitigate Spurious Correlations in Natural Language Inference Datasets (2022.acl-long)
Copied to clipboard
| Challenge: | Natural language processing models exploit spurious correlations between features and labels in datasets to perform well only within the distributions they are trained on. |
| Approach: | They propose to generate a debiased version of a dataset and replace it with training data to train a model that is generalised to different task distributions. |
| Outcome: | The proposed method outperforms or performs comparable to state-of-the-art debiasing strategies on a large suite of debiased, out-of distribution, and adversarial test sets. |
Decorrelate Irrelevant, Purify Relevant: Overcome Textual Spurious Correlations from a Feature Perspective (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods to debiase samples with biased features obstructs the model in learning from non-biased parts of the samples. |
| Approach: | They propose to eliminate spurious correlations in a fine-grained manner from a feature space perspective by using Random Fourier Features and weighted re-sampling to decorrelate dependencies between features. |
| Outcome: | The proposed method eliminates spurious correlations in a fine-grained manner from a feature space perspective. |
Identifying and Mitigating Spurious Correlations for Improving Robustness in NLP Models (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing work identifies task-specific shortcuts via human priors or error analyses, which requires extensive expertise and efforts. |
| Approach: | They propose to automatically identify spurious correlations in NLP models at scale by using existing interpretability methods to extract tokens that significantly affect model’s decision process. |
| Outcome: | The proposed method can identify spurious correlations in NLP models at scale and mitigate these leads to more robust models in multiple applications. |
Investigating Ensemble Methods for Model Robustness Improvement of Text Classifiers (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to reduce model's reliance on bias features ignore the learnability of these features. |
| Approach: | They propose to reduce models' reliance on bias features by first training models with fixed low-capacity models which ignore the learnability of the bias features. |
| Outcome: | The proposed models can perform better on out-of-distribution datasets than baseline models with a more sophisticated model design. |
Unlearn Dataset Bias in Natural Language Inference by Fitting the Residual (D19-61)
Copied to clipboard
| Challenge: | Statistical natural language inference models are susceptible to learning dataset bias. |
| Approach: | They propose a debiasing algorithm that debiases models that use only known dataset biases . they use two benchmark datasets to train three high-performing NLI models . |
| Outcome: | The proposed learning objective improves model performance on challenge datasets while maintaining reasonable performance on original datasets. |
Towards Robustifying NLI Models Against Lexical Dataset Biases (2020.acl-main)
Copied to clipboard
| Challenge: | Recent studies show that deep learning models exploit dataset biases without deep understanding of the language semantics. |
| Approach: | They propose two methods to debiase models against lexical dataset biases . they use contradiction-word bias and word-overlapping bias as examples . |
| Outcome: | The proposed method removes label bias at embedding level, while the other uses a bag-of-words sub-model to capture features likely to exploit the bias. |
Influence Tuning: Demoting Spurious Correlations via Instance Attribution and Instance-Driven Updates (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to interpret black-box models to learn spurious correlations are not well understood. |
| Approach: | They propose a procedure that leverages model interpretations to update parameters towards a plausible interpretation rather than an interpretation that relies on spurious patterns in data. |
| Outcome: | The proposed procedure outperforms baseline methods that use adversarial training in a controlled setup. |