Fine-Tuned LLMs are “Time Capsules” for Tracking Societal Bias Through Books (2025.naacl-long)
Copied to clipboard
| Challenge: | We develop a corpus comprising 593 fictional books across seven decades (1950-2019) to track bias evolution. |
| Approach: | They develop a method to trace and quantify bias evolution using fine-tuned LLMs on fictional books across seven decades to track bias evolution. |
| Outcome: | The proposed method traces and quantifies bias evolution in a corpus of 593 fictional books across seven decades. |
Similar Papers
Quantifying Generative Media Bias with a Corpus of Real-world and Generated News Articles (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies focus on LLMs undertaking political questionnaires, which offers only limited insights into their biases and operational nuances. |
| Approach: | They propose to use a curated dataset to generate 56,700 synthetic articles using nine LLMs. |
| Outcome: | The proposed model can detect political biases using supervised models and LLMs. |
LLMs are Biased Teachers: Evaluating LLM Bias in Personalized Education (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies have shown that relying on LLMs as information providers may hurt student learning. |
| Approach: | They introduce and apply two bias score metrics to evaluate LLMs for bias in the personalized educational setting, specifically on the models’ roles as “teachers.” |
| Outcome: | The proposed models harm student learning by perpetuating harmful stereotypes and reversing them. |
OpinionGPT: Modelling Explicit Biases in Instruction-Tuned LLMs (2024.naacl-demo)
Copied to clipboard
| Challenge: | Current research seeks to de-bias such models, or suppress potentially biased answers. |
| Approach: | They present a web demo to test the biases of instruction-tuned Large Language Models . they identify 11 different biase based on a corpus of data . |
| Outcome: | The proposed demo shows that biases in instruction-tuning are explicit and transparent . the demo shows how the model was trained and showcases the web application . |
Investigating Bias in LLM-Based Bias Detection: Disparities between LLMs and Human Perception (2025.coling-main)
Copied to clipboard
| Challenge: | Detecting media bias is critical due to the spread of misinformation and disinformation on social media platforms. |
| Approach: | They investigate the presence and nature of bias within large language models and its consequential impact on media bias detection. |
| Outcome: | The proposed debiasing strategies include prompt engineering and model fine-tuning. |
Demographic-Aware Language Model Fine-tuning as a Bias Mitigation Technique (2022.aacl-short)
Copied to clipboard
| Challenge: | In this paper, we analyze the variations in gender and racial biases in BERT-like language models when exposed to different demographic groups. |
| Approach: | They analyze gender and racial biases in BERT-like language models when exposed to different demographic groups. |
| Outcome: | The proposed model can mitigate biases in text authored by disadvantaged demographic groups compared to advantaged groups . the proposed model is agnostic to the language of the speakers behind the language . |
LLM Tropes: Revealing Fine-Grained Values and Opinions in Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to evaluate latent values and opinions in large language models suffer from three notable shortcomings. |
| Approach: | They propose to analyze 156k LLM responses to 62 propositions of the Political Compass Test (PCT) generated by 6 LLMs using 420 prompt variations. |
| Outcome: | The proposed analysis of 156k LLM responses to the Political Compass Test (PCT) generated by 6 LLMs shows that tropes are recurrent and consistent across prompts. |
Neutral Is Not Unbiased: Evaluating Implicit and Intersectional Identity Bias in LLMs Through Structured Narrative Scenarios (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models often reproduce societal biases, yet most evaluations overlook how such biase evolve across nuanced contexts or intersecting identities. |
| Approach: | They propose a scenario-based evaluation framework built on 100 narrative tasks . they use critical discourse analysis and quantitative linguistic metrics to analyze LLMs . |
| Outcome: | The proposed evaluation framework provides ethically coherent and socially plausible settings for probing model behavior. |
Systematic Biases in LLM Simulations of Debates (2024.emnlp-main)
Copied to clipboard
| Challenge: | Current research suggests that LLM-based agents become increasingly human-like in their performance, sparking interest in using these AI agents as substitutes for human participants in behavioral studies. |
| Approach: | They propose to use LLMs to simulate political debates on topics that are important aspects of people’s day-to-day lives and decision-making processes. |
| Outcome: | The proposed model can simulate political debates on topics that are important aspects of people’s day-to-day lives and decision-making processes. |
Confronting LLMs with Traditional ML: Rethinking the Fairness of Large Language Models in Tabular Classifications (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent studies suggest using large language models to make tabular classifications . however, LLMs have been shown to exhibit harmful social biases based on stereotypes and inequalities present in society. |
| Approach: | They propose to use large language models to make tabular classifications . they show that LLMs inherit biases from their training data . |
| Outcome: | The proposed models exhibit harmful biases that reflect stereotypes and inequalities in society. |
Towards Implicit Bias Detection and Mitigation in Multi-Agent LLM Interactions (2024.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that large language models are susceptible to societal biases due to their exposure to human-generated data. |
| Approach: | They propose two strategies to mitigate implicit gender biases in large language models . they create scenarios where implicit gender is present and develop a metric to assess the presence of biase . |
| Outcome: | The proposed methods mitigate implicit biases with self-reflection and fine-tuning. |