Where Fact Ends and Fairness Begins: Redefining AI Bias Evaluation through Cognitive Biases (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing benchmarks conflate factual correctness and normative fairness . a model may generate responses that are factually accurate but socially unfair . |
| Approach: | They propose a benchmark to examine the boundary between fact and fair . they draw on representativeness bias, attribution bias and ingroup–outgroup bias to explain why models often misalign fact and faireness. |
| Outcome: | The proposed model is based on ten frontier models and is available on github . it is compared with a standard model that generates people of color in Nazi-era uniforms . |
Similar Papers
Fair Enough: Standardizing Evaluation and Model Selection for Fairness Research in NLP (2023.eacl-main)
Copied to clipboard
| Challenge: | Modern NLP systems exhibit a range of biases, which a growing literature on model debiasing attempts to correct. |
| Approach: | They propose to clarify the current situation and plot a course for meaningful progress in fair learning by making clear inter-relations among the current gamut of methods and their relation to fairness theory. |
| Outcome: | The proposed approach addresses the practical problem of model selection, which involves a trade-off between fairness and accuracy and has led to systemic issues in fairness research. |
Bias and Fairness in Natural Language Processing (D19-2)
Copied to clipboard
| Challenge: | a tutorial will review the history of bias and fairness studies in machine learning and language processing . |
| Approach: | This tutorial reviews the history of bias and fairness studies in machine learning and language processing . it presents recent community effort to quantify and mitigat bias in natural language processing models . |
| Outcome: | This tutorial reviews the history of bias and fairness studies in machine learning and language processing . it aims to quantify and mitigate bias in natural language processing models for a wide spectrum of tasks . |
F²Bench: An Open-ended Fairness Evaluation Benchmark for LLMs with Factuality Considerations (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing fairness evaluation benchmarks for large language models rely on closed-ended evaluation formats that overlook factuality considerations rooted in historical, social, physiological, and cultural contexts. |
| Approach: | They propose an open-ended fairness evaluation benchmark for large language models . they incorporate factuality considerations and multi-turn reasoning into the benchmark . |
| Outcome: | The proposed benchmark incorporates factual grounding and text generation to better reflect the complexities of real-world model usage. |
The Impossibility of Fair LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Existing frameworks for evaluating large language models do not extend to general-purpose AI contexts or are infeasible in practice. |
| Approach: | They analyze a variety of technical fairness frameworks to find inherent challenges . they find that each framework does not logically extend to the general-purpose AI context . |
| Outcome: | The proposed frameworks do not logically extend to the general-purpose AI context or are infeasible in practice due to large amounts of unstructured training data and potential combinations of human populations, use cases, and sensitive attributes. |
Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metrics (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing fairness metrics quantify the differences in a model’s behaviour across a range of demographic groups. |
| Approach: | They propose to unify existing fairness metrics and compare them to three generalized fairness measures to reveal the connections between them. |
| Outcome: | The proposed measures can be explained by differences in parameter choices, and the results are consistent with previous studies. |
Whose Preferences? Differences in Fairness Preferences and Their Impact on the Fairness of AI Utilizing Human Feedback (2024.acl-long)
Copied to clipboard
| Challenge: | a growing body of work on learning from human feedback to align various aspects of machine learning systems with human values and preferences is focusing on the setting of fairness in content moderation. |
| Approach: | They propose to use human feedback to determine how two comments should be treated in content moderation to learn about human values and preferences. |
| Outcome: | The proposed approach is promising, as human preferences can often not be A: Some ladies like smaller men. B: Some men like smaller guys. Figure 1 shows that the proposed approach performs better for demographic intersections than a single classifier that gives equal weight to each annotation. |
Re-contextualizing Fairness in NLP: The Case of India (2022.aacl-main)
Copied to clipboard
| Challenge: | Recent research has revealed undesirable biases in NLP data and models . however, these efforts focus of social disparities in the West and are not directly portable to other geo-cultural contexts. |
| Approach: | They propose a framework to re-contextualize NLP fairness research for the Indian context . they build resources for fairness evaluation in the Indian and delve deeper into social stereotypes for Region and Religion . |
| Outcome: | The proposed framework can be generalized to other geo-cultural contexts. |
Benchmarking Intersectional Biases in NLP (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent work on fairness of machine learning models has focused on how to debias, but research on the fairness and performance of biased/debiased models on downstream prediction tasks has been limited. |
| Approach: | They assess intersectional bias - fairness across multiple demographic dimensions . they highlight possible causes and make recommendations for future NLP debiasing research. |
| Outcome: | The proposed approaches fare well in terms of fairness-accuracy trade-off, but are unable to effectively alleviate bias in downstream tasks. |
Evaluating and Characterizing Human Rationales (2020.emnlp-main)
Copied to clipboard
| Challenge: | a new study examines how human rationales perform on automatic metrics . human-generated rationale evaluation is difficult because of its ambiguity . |
| Approach: | They propose to use model-dependent baseline performance to evaluate rationale quality . they propose to also use "fidelity curves" to reveal properties such as irrelevance and redundancy . |
| Outcome: | The proposed methods characterize rationale quality based on model retraining and using "fidelity curves" the proposed methods lead to actionable suggestions for evaluating and characterizing rationales . |
R-Fairness: Assessing Fairness of Ranking in Subjective Data (2025.acl-long)
Copied to clipboard
| Challenge: | Subjective data, reflecting individual opinions, permeates platforms like Yelp and Amazon . despite the prevalence of such platforms, little attention has been given to fairness in their context . |
| Approach: | They propose a fairness assessment pipeline that starts with data collection phase and then iterates through rated items. |
| Outcome: | The proposed approach favors groups writing best-ranked reviews over others on collaborative rating platforms. |