Robust Natural Language Understanding with Residual Attention Debiasing (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing ensemble-based debiasing methods do not address unintended dataset biases . attention plays a crucial role in providing robust prediction in NLU models . |
| Approach: | They propose an end-to-end debiasing method that mitigates unintended biases from attention. |
| Outcome: | The proposed method improves the OOD performance of BERT-based models on three benchmarks. |
Similar Papers
End-to-End Self-Debiasing Framework for Robust NLU Training (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing models incorporate dataset biases leading to strong performance on in-distribution test sets but poor performance on out-of-distortion (OOD) tests. |
| Approach: | They propose a debiasing framework where the shallow representations of the main model are used to derive a bias model and both models are trained simultaneously. |
| Outcome: | The proposed framework outperforms existing approaches on three well-studied NLU tasks while still delivering high in-distribution performance. |
Debiasing Methods in Natural Language Understanding Make Bias More Accessible (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent debiasing methods in natural language understanding improve performance on out-of-distribution datasets by pressuring models into making unbiased predictions. |
| Approach: | They propose a general probing-based framework that allows for post-hoc interpretation of biases in language models and use an information-theoretic approach to measure the extractability of certain biase . |
| Outcome: | The proposed framework allows for post-hoc interpretation of biases in language models and measures the extractability of certain biase . |
Towards Stable Natural Language Understanding via Information Entropy Guided Debiasing (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to debiase Natural Language Understanding models use dataset biases instead of learning the intended task. |
| Approach: | They propose a debiasing framework that detects and purifies dataset biases using information entropy. |
| Outcome: | The proposed framework improves the stability of performance on out-of-distribution datasets for a set of widely adopted NLU models. |
End-to-End Bias Mitigation by Modelling Biases in Corpora (2020.acl-main)
Copied to clipboard
| Challenge: | Recent studies have shown that strong natural language understanding models are prone to relying on unwanted dataset biases without learning the underlying task. |
| Approach: | They propose two learning strategies to train neural models that are more robust to dataset biases and transfer better to out-of-domain datasets. |
| Outcome: | The proposed methods improve robustness in all settings and transfer better to out-of-domain datasets. |
Mind the Trade-off: Debiasing NLU Models without Degrading the In-distribution Performance (2020.acl-main)
Copied to clipboard
| Challenge: | Recent studies show that pre-trained language models rely heavily on idiosyncratic biases of datasets. |
| Approach: | They propose a method which discourages models from exploiting biases while enabling them to receive enough incentive to learn from all the training examples. |
| Outcome: | The proposed method improves on out-of-distribution datasets while maintaining original in-district accuracy. |
Avoiding the Hypothesis-Only Bias in Natural Language Inference via Ensemble Adversarial Training (2020.emnlp-main)
Copied to clipboard
| Challenge: | Neural models pick up on annotation artefacts and spurious correlations, resulting in learning sentences that suffer from the same biases. |
| Approach: | They propose to tackle this problem by using adversarial training to reduce the bias in sentence representations by using an ensemble of adversaries. |
| Outcome: | The proposed approach produces more robust models outperforming previous de-biasing efforts when generalised to 12 other NLI datasets. |
Towards Robustifying NLI Models Against Lexical Dataset Biases (2020.acl-main)
Copied to clipboard
| Challenge: | Recent studies show that deep learning models exploit dataset biases without deep understanding of the language semantics. |
| Approach: | They propose two methods to debiase models against lexical dataset biases . they use contradiction-word bias and word-overlapping bias as examples . |
| Outcome: | The proposed method removes label bias at embedding level, while the other uses a bag-of-words sub-model to capture features likely to exploit the bias. |
Unlearn Dataset Bias in Natural Language Inference by Fitting the Residual (D19-61)
Copied to clipboard
| Challenge: | Statistical natural language inference models are susceptible to learning dataset bias. |
| Approach: | They propose a debiasing algorithm that debiases models that use only known dataset biases . they use two benchmark datasets to train three high-performing NLI models . |
| Outcome: | The proposed learning objective improves model performance on challenge datasets while maintaining reasonable performance on original datasets. |
InterFair: Debiasing with Natural Language Feedback for Fair Interpretable Predictions (2023.emnlp-main)
Copied to clipboard
| Challenge: | Debiasing methods in NLP models focus on isolating information related to a sensitive attribute (e.g., gender or race) but instead argue that a favorable debiaser should use sensitive information ‘fairly,’ with explanations, rather than blindly eliminating it. |
| Approach: | They propose that a favorable debiasing method should use sensitive information ‘fairly,’ with explanations, rather than blindly eliminating it. |
| Outcome: | The proposed approach reduces bias in explanations while maintaining the same prediction accuracy. |
Towards Debiasing NLU Models from Unknown Biases (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent proposed debiasing methods rely on the assumption that the types of bias should be known a-priori, which limits their application to many NLU tasks and datasets. |
| Approach: | They propose a framework that prevents models from mainly utilizing biases without knowing them in advance. |
| Outcome: | The proposed framework allows existing methods to retain performance improvement on challenge datasets without specifically targeting biases. |