Text Style Transfer for Bias Mitigation using Masked Language Modeling (2022.naacl-srw)
Copied to clipboard
| Challenge: | Various research findings have concluded that biased textual data has significant effects on target demographic groups. |
| Approach: | They propose a text-style transfer model that can be trained on non-parallel data and be used to automatically mitigate bias in textual data. |
| Outcome: | The proposed model improves on limitations of existing methods while maintaining good style transfer accuracy. |
Similar Papers
Text Style Transferring via Adversarial Masking and Styled Filling (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for text style transfer suffer from two challenges: the word masking procedure may mistakenly remove unexpected words and the selected words in the word filling procedure lack diversity and semantic consistency. |
| Approach: | They propose a style transfer model with adversarial masking and styled filling techniques to solve these challenges. |
| Outcome: | The proposed model performs well on two benchmark text style transfer data sets. |
Fighting Offensive Language on Social Media with Unsupervised Text Style Transfer (P18-2)
Copied to clipboard
| Challenge: | Existing methods to tackle the problem of offensive language in social media are based on machine learning. |
| Approach: | They propose a method for training encoder-decoders using non-parallel data . they use a collaborative classifier, attention and the cycle consistency loss . |
| Outcome: | The proposed method outperforms state-of-the-art text style transfer systems on Twitter and Reddit . it produces reliable non-offensive transferred sentences, the authors show . |
TextSETTR: Few-Shot Text Style Extraction and Tunable Targeted Restyling (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for text style transfer require style-labeled training data, but use only labeled data at inference time. |
| Approach: | They propose a method that uses readily-available unlabeled text to train style transfer . they use a style vector to condition a decoder to perform style transfer using unlabelled text . |
| Outcome: | The proposed method is competitive on sentiment transfer, even compared to models trained fully on labeled data. |
Unmasking Style Sensitivity: A Causal Analysis of Bias Evaluation Instability in Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods to assess social biases in natural language processing models show unexpected instability when input texts undergo minor stylistic changes. |
| Approach: | They conduct a comprehensive analysis of how style transformations impact bias evaluation results . they find formal style transformation significantly affects bias scores . larger models show greater sensitivity to stylistic variations, they find . |
| Outcome: | The proposed method fails to detect appearance bias, sexual orientation bias, religious bias and religious bias in large language models. |
Disentangled Representation Learning for Non-Parallel Text Style Transfer (P19-1)
Copied to clipboard
| Challenge: | a paper aims to disentangle latent representations of style and content in language models . auxiliary multi-task and adversarial objectives are used to disentangle the latent space . |
| Approach: | They propose a simple yet effective approach to disentangling latent representations . they propose auxiliary multi-task and adversarial objectives to disentangle style and content . |
| Outcome: | The proposed approach achieves high performance in terms of transfer accuracy, content preservation, and language fluency compared to previous approaches . |
End-to-End Bias Mitigation by Modelling Biases in Corpora (2020.acl-main)
Copied to clipboard
| Challenge: | Recent studies have shown that strong natural language understanding models are prone to relying on unwanted dataset biases without learning the underlying task. |
| Approach: | They propose two learning strategies to train neural models that are more robust to dataset biases and transfer better to out-of-domain datasets. |
| Outcome: | The proposed methods improve robustness in all settings and transfer better to out-of-domain datasets. |
Style Transfer Through Back-Translation (P18-1)
Copied to clipboard
| Challenge: | a new method for automatic style transfer is proposed to preserve the meaning of the text while reducing stylistic properties. |
| Approach: | They propose a method for automatic style transfer that uses latent representations of the input sentence to preserve meaning while reducing stylistic properties. |
| Outcome: | The proposed method improves on sentiment, gender and political slant styles on three different styles. |
Upstream Mitigation Is Not All You Need: Testing the Bias Transfer Hypothesis in Pre-Trained Language Models (2022.acl-long)
Copied to clipboard
| Challenge: | Large language models and other massively pre-trained "foundation" models can easily adapt to a wide variety of downstream tasks in a process called finetuning. |
| Approach: | They propose to use the bias transfer hypothesis to reduce social biases internalized by large language models during pre-training into harmful task-specific behavior after fine-tuning. |
| Outcome: | The bias transfer hypothesis is the theory that social biases internalized by large language models during pre-training transfer into harmful task-specific behavior after fine-tuning. |
Bias and Fairness in Natural Language Processing (D19-2)
Copied to clipboard
| Challenge: | a tutorial will review the history of bias and fairness studies in machine learning and language processing . |
| Approach: | This tutorial reviews the history of bias and fairness studies in machine learning and language processing . it presents recent community effort to quantify and mitigat bias in natural language processing models . |
| Outcome: | This tutorial reviews the history of bias and fairness studies in machine learning and language processing . it aims to quantify and mitigate bias in natural language processing models for a wide spectrum of tasks . |
Unsupervised Text Style Transfer with Padded Masked Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for style transfer are difficult to obtain and require substantial amounts of parallel training examples to work well. |
| Approach: | They propose an unsupervised method for style transfer that uses masked language models to find the text spans where the two models disagree the most in terms of likelihood. |
| Outcome: | The proposed method performs competitively in a fully unsupervised setting and improves accuracy in low-resource settings by over 10 percentage points when pre-training on silver training data generated by Masker. |