| Challenge: | Existing adversarial methods only partially mitigate the problem of model bias, added to which their training procedures are unstable. |
| Approach: | They propose a method where discriminators are encouraged to learn orthogonal hidden representations from one another to reduce model bias. |
| Outcome: | The proposed method significantly reduces bias and stability of training over standard methods. |
Similar Papers
Deep Adversarial Learning for NLP (N19-5)
Copied to clipboard
| Challenge: | Adversarial learning is a game-theoretic learning paradigm that has achieved huge successes in the field of Computer Vision recently. |
| Approach: | This tutorial introduces the foundations of deep adversarial learning and some practical problems and solutions in NLP. |
| Outcome: | This tutorial introduces the foundations of deep adversarial learning and some practical problems and solutions in NLP. |
Avoiding the Hypothesis-Only Bias in Natural Language Inference via Ensemble Adversarial Training (2020.emnlp-main)
Copied to clipboard
| Challenge: | Neural models pick up on annotation artefacts and spurious correlations, resulting in learning sentences that suffer from the same biases. |
| Approach: | They propose to tackle this problem by using adversarial training to reduce the bias in sentence representations by using an ensemble of adversaries. |
| Outcome: | The proposed approach produces more robust models outperforming previous de-biasing efforts when generalised to 12 other NLI datasets. |
Contrastive Learning as a Polarizer: Mitigating Gender Bias by Fair and Biased sentences (2024.findings-naacl)
Copied to clipboard
| Challenge: | Recent studies have highlighted social biases inherent in training data can lead models to learn and propagate them. |
| Approach: | They propose a contrastive learning method that uses anchor points to push further negatives and pull closer positives within the representation space. |
| Outcome: | The proposed method achieves state-of-the-art in the ICAT score on the StereoSet, a benchmark for measuring bias in models. |
When and Why Does Bias Mitigation Work? (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Neural models exploit shallow surface features to perform language understanding tasks, rather than learning the deeper language understanding and reasoning skills that practitioners desire. |
| Approach: | They propose to use model debiasing techniques to pressure models away from spurious features and to use them to learn useful representations instead. |
| Outcome: | The proposed methods increase models' reliance on hidden biases instead of learning robust features that help them solve a task. |
End-to-End Bias Mitigation by Modelling Biases in Corpora (2020.acl-main)
Copied to clipboard
| Challenge: | Recent studies have shown that strong natural language understanding models are prone to relying on unwanted dataset biases without learning the underlying task. |
| Approach: | They propose two learning strategies to train neural models that are more robust to dataset biases and transfer better to out-of-domain datasets. |
| Outcome: | The proposed methods improve robustness in all settings and transfer better to out-of-domain datasets. |
It’s Morphin’ Time! Combating Linguistic Discrimination with Inflectional Perturbations (2020.acl-main)
Copied to clipboard
| Challenge: | Existing work on societal bias in NLP focuses on race and gender . linguistic background is a unique attribute that has been largely ignored in the field . |
| Approach: | They examine linguistic background to craft plausible adversarial examples that expose biases in popular NLP models. |
| Outcome: | The proposed model improves robustness without sacrificing performance on clean data. |
Data Augmentation with Adversarial Training for Cross-Lingual NLI (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to train cross-lingual models with labeled data are subpar, resulting in subpar results. |
| Approach: | They propose a data augmentation strategy that enriches data to reflect more diversity in a semantically faithful way and leverages adversarial training regimens to achieve greater robustness. |
| Outcome: | The proposed approach improves cross-lingual inference by leveraging the data to reflect more diversity in a semantically faithful way. |
Revisiting Adversarial Autoencoder for Unsupervised Word Translation with Cycle Consistency and Improved Training (N19-1)
Copied to clipboard
| Challenge: | Recent work has shown superior performance for non-adversarial methods in more challenging language pairs. |
| Approach: | They propose to use adversarial autoencoder to map monolingual embeddings to a shared space and to put the target encoders as an adversary against the corresponding discriminator. |
| Outcome: | The proposed method is more robust and achieves better performance than previously proposed adversarial and non-adversarial methods. |
Balancing out Bias: Achieving Fairness Through Balanced Training (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to reducing group bias do not account for correlations between author demographics and linguistic variables, limiting their effectiveness. |
| Approach: | They extend a method for countering group bias using balanced training by balancing each demographic group in training and using protected attributes as input. |
| Outcome: | The proposed model outperforms all other methods when combined with balanced training. |
Mitigating Gender Bias for Neural Dialogue Generation with Adversarial Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent research shows that dialogue systems trained on human conversation data are biased and can produce responses that reflect people’s gender prejudice. |
| Approach: | They propose a novel adversarial learning framework Debiased-Chat to train dialogue models free from gender bias while keeping their performance. |
| Outcome: | The proposed framework significantly reduces gender bias in dialogue models while maintaining the response quality. |