COFFEE: Counterfactual Fairness for Personalized Text Generation in Explainable Recommendation (2023.emnlp-main)
Copied to clipboard
Nan Wang, Qifan Wang, Yi-Chia Wang, Maziar Sanjabi, Jingzhou Liu, Hamed Firooz, Hongning Wang, Shaoliang Nie
| Challenge: | Personalized text generation (PTG) is a key component of our digital lives but can inadvertently associate different levels of linguistic quality with users’ protected attributes. |
| Approach: | They propose a framework to achieve measure-specific counterfactual fairness in explanation generation by focusing on one of the most studied settings: generating natural language explanations for recommendations. |
| Outcome: | The proposed framework achieves measure-specific counterfactual fairness in explanation generation. |
Similar Papers
CFL: Causally Fair Language Models Through Token-level Attribute Controlled Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to control attributes of Language Models (LMs) for text generation are not safe, as toxicity and bias goals are opposed to each other. |
| Approach: | They propose a method to control the attributes of Language Models (LMs) for the text generation task using Causal Average Treatment Effect (ATE) scores and counterfactual augmentation. |
| Outcome: | The proposed architecture achieves state of the art performance for toxic degeneration, which are computed using Real Toxicity Prompts. |
Improving Personalized Explanation Generation through Visualization (2022.acl-long)
Copied to clipboard
| Challenge: | Existing explainable recommendation models generate repetitive sentences for different items or empty sentences with insufficient details. |
| Approach: | They propose a visual-enhanced approach to generate rating scores and text explanations using visualization generation and text–image matching discrimination. |
| Outcome: | The proposed approach improves both the text quality and the diversity and explainability of the generated explanations. |
Learning to Generate Equitable Text in Dialogue from Biased Training Data (2023.acl-long)
Copied to clipboard
| Challenge: | Absence of equitable and inclusive principles can hinder the formation of common ground, which in turn negatively impacts the overall performance of the system. |
| Approach: | They propose to use theories of computational learning to study equitable text generation in dialogues using augmented data to prove formal definitions of equity in text generation and formal connections between human-likeness and learning equity. |
| Outcome: | The proposed model predicts relative-performance of multiple algorithms in generating equitable text as measured by human and automated evaluation. |
Reducing Sentiment Bias in Language Models via Counterfactual Evaluation (2020.findings-emnlp)
Copied to clipboard
Po-Sen Huang, Huan Zhang, Ray Jiang, Robert Stanforth, Johannes Welbl, Jack Rae, Vishal Maini, Dani Yogatama, Pushmeet Kohli
| Challenge: | Language modeling has advanced rapidly due to efficient model architectures and the availability of large text corpora. |
| Approach: | They propose to embed and regularize sentiment prediction-derived regularizations on the language model’s latent representations to reduce bias in the sentiment of generated text. |
| Outcome: | The proposed methods reduce bias in the sentiment of generated text by adopting individual and group fairness metrics from the fair machine learning literature. |
FairPrism: Evaluating Fairness-Related Harms in Text Generation (2023.acl-long)
Copied to clipboard
Eve Fleisig, Aubrie Amstutz, Chad Atalla, Su Lin Blodgett, Hal Daumé III, Alexandra Olteanu, Emily Sheng, Dan Vann, Hanna Wallach
| Challenge: | FairPrism dataset provides a framework for measuring and mitigating fairness-related harms caused by AI text generation systems. |
| Approach: | They propose a dataset of 5,000 examples of AI-generated English text with detailed human annotations covering a diverse set of harms relating to gender and sexuality. |
| Outcome: | FairPrism is a dataset of 5,000 examples of AI-generated English text with detailed human annotations covering harms relating to gender and sexuality. |
Your fairness may vary: Pretrained language model fairness in toxic text classification (2022.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained, bidirectional language models have revolutionized natural language processing research . authors show that focusing on accuracy measures alone can lead to models with wide variation in fairness characteristics . |
| Approach: | They propose to use two post-processing methods to improve model fairness without retraining . they use pretrained language models of varying sizes on two toxic text classification tasks . |
| Outcome: | The proposed methods improve model fairness without retraining . the results show that the fairness variation is more than just accuracy . |
A Survey on Natural Language Counterfactual Generation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in NLP are driven by a variety of Large Language Models (LLMs), such as GPT-3 (175B) and PaLM (540B). |
| Approach: | They propose a taxonomy that categorizes the methods into four groups and summarizes the metrics for evaluating the generation quality. |
| Outcome: | The proposed taxonomy categorizes the generation methods into four groups and summarizes the metrics for evaluating the quality. |
A Prompt Array Keeps the Bias Away: Debiasing Vision-Language Models with Adversarial Learning (2022.aacl-main)
Copied to clipboard
| Challenge: | Large-scale, pretrained vision-language models are growing in popularity due to impressive performance on downstream tasks with minimal finetuning. |
| Approach: | They propose to apply ranking metrics to image-text representations to investigate bias measures and debiasing methods to reduce various bias measures. |
| Outcome: | The proposed model reduces bias measures with minimal degradation to image-text representations. |
Debiasing Text Safety Classifiers through a Fairness-Aware Ensemble (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Increasing use of large language models (LLMs) require performant guardrails to ensure the safety of inputs and outputs . when these guardrail are trained on imbalanced data, they can learn the societal biases resulting from the model's performance. |
| Approach: | They propose a method for mitigating counterfactual fairness in closed-source text safety classifiers by using a debiasing regularizer and a threshold-agnostic metric. |
| Outcome: | The proposed method outperforms classifiers and acts as a debiasing regularizer . it uses threshold-agnostic metrics and Fair Data Reweighting (FDW) to assess the counterfactual fairness of a model . |
The Woman Worked as a Babysitter: On Biases in Language Generation (D19-1)
Copied to clipboard
| Challenge: | a systematic study of biases in natural language generation (NLG) is presented . a study of language models in NLG is conducted by examining language models. |
| Approach: | They propose a systematic study of biases in natural language generation by analyzing text generated from prompts that contain mentions of different demographic groups. |
| Outcome: | The proposed method reveals biases in natural language generation (NLG) by analyzing text generated from demographic prompts. |