On the Relationship between Truth and Political Bias in Language Models (2024.emnlp-main)
Copied to clipboard
Suyash Fulay, William Brannon, Shrestha Mohanty, Cassandra Overney, Elinor Poole-Dayan, Deb Roy, Jad Kabbara
| Challenge: | Language model alignment research often attempts to ensure that models are helpful and harmless, but can obscure how improving one aspect might impact the other. |
| Approach: | They analyze the relationship between truthfulness and political bias in language models. |
| Outcome: | The results show that optimizing models for truthfulness results in a left-leaning political bias. |
Similar Papers
Whose Emotions and Moral Sentiments do Language Models Reflect? (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing research has focused on positional alignment, which measures how closely the models mimic the opinions and stances of different social groups. |
| Approach: | They define the problem of affective alignment, which measures how LMs’ emotional and moral tone represents those of different groups. |
| Outcome: | The results show that the models represent the perspectives of some social groups better than others, suggesting a systemic bias within LMs. |
PolBiX: Detecting LLMs’ Political Bias in Fact-Checking through X-phemisms (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a few models show tendencies of political bias, but this is not mitigated by explicitly calling for objectivism in prompts. |
| Approach: | They investigate political bias by exchanging words with euphemisms or dysphemismas in German claims. |
| Outcome: | The proposed model shows that political bias influences truthfulness assessment more than political leaning . |
The Hidden Bias: A Study on Explicit and Implicit Political Stereotypes in Large Language Models (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly integral to information dissemination and decision-making processes. |
| Approach: | They investigate political bias and stereotype propagation across eight prominent LLMs using the two-dimensional Political Compass Test. |
| Outcome: | The political bias and stereotype propagation of large language models is investigated using the two-dimensional Political Compass Test (PCT) key findings reveal a left-leaning political alignment across all investigated models. |
From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models (2023.acl-long)
Copied to clipboard
| Challenge: | Hundreds of studies have highlighted ethical issues in NLP models . |
| Approach: | They propose to measure media biases in LMs trained on diverse data sources . they focus on hate speech and misinformation detection . |
| Outcome: | The proposed methods quantify the fairness of downstream NLP models trained on politically biased LMs. |
Analyzing Political Bias in LLMs via Target-Oriented Sentiment Classification (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to analyze political biases rely on small-size intermediate tasks and the LLMs themselves. |
| Approach: | They propose an entropy-based inconsistency metric to encode political biases . they insert 1319 demographically and politically diverse politician names in 450 political sentences . |
| Outcome: | The proposed method combines high accuracy with a correct understanding of the candidate candidate. |
Democratic or Authoritarian? Probing a New Dimension of Political Biases in Large Language Models (2026.eacl-long)
Copied to clipboard
| Challenge: | Prior work on LLM biases focused on socio-demographic and left–right political dimensions, but little attention has been paid to how they align with broader geopolitical value systems. |
| Approach: | They propose a method to assess how LLMs align with broader geopolitical value systems, particularly the democracy–authoritarianism spectrum. |
| Outcome: | The proposed method combines the F-scale, FavScore and role-model probing to assess which figures are cited as general role models by LLMs. |
How Gender Interacts with Political Values: A Case Study on Czech BERT Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Neural language models are trained on large text corpora that contain value-burdened content and often capture undesirable biases, which the models reflect. |
| Approach: | They propose a method to measure the model's perceived political values by comparing Czech with a representative value survey. |
| Outcome: | The proposed method does not assign statement probability following value-driven reasoning and there is no systematic difference between feminine and masculine sentences. |
Measuring Political Bias in Large Language Models: What Is Said and How It Is Said (2024.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks and measures focus on gender and racial biases, but political bias exists in LLMs and can lead to polarization and other harms in downstream applications. |
| Approach: | They propose to analyze the content and style of LLMs generated by political issues and propose a framework that can be scalable to other topics. |
| Outcome: | The proposed framework is easily scalable to other topics and is explainable. |
Bias in the East, Bias in the West: A Bilingual Analysis of LLM Political Bias on U.S.- and China-Related Issues (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large language models (LLMs) can exhibit political biases, which creates a risk of undue influence on LLM users and public opinion. |
| Approach: | They use a dataset of 36k real-time test prompts to measure LLM political bias on U.S. and Chinese issues. |
| Outcome: | The proposed model origin and prompt language influence bias on 60 political issues. |
Only a Little to the Left: A Theory-grounded Measure of Political Bias in Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Political biases in language models can affect performance in many applications . political biased models are often left-leaning, but are generally more left- leaning for instruction-tuned models . |
| Approach: | They propose to use the Political Compass Test to measure political bias in language models . they use survey-based evaluation tools to test prompts and classify their political stances . |
| Outcome: | The proposed model is based on the Political Compass Test, but is not scientifically valid. |