People will agree what I think: Investigating LLM’s False Consensus Effect (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent studies have focused on the False Consensus Effect (FCE) where individuals overestimate the extent to which others share their beliefs or behaviors. |
| Approach: | They conduct two studies to examine the FCE phenomenon in Large Language Models (LLMs) they find that popular LLMs have FCE and that they have different prompting styles. |
| Outcome: | The proposed model is popular among LLM users and specifies the conditions when FCE becomes more or less prevalent compared to normal usage. |
Similar Papers
Biased LLMs can Influence Political Decision-Making (2025.acl-long)
Copied to clipboard
Jillian Fisher, Shangbin Feng, Robert Aron, Thomas Richardson, Yejin Choi, Daniel W Fisher, Jennifer Pan, Yulia Tsvetkov, Katharina Reinecke
| Challenge: | Recent studies have found that biased LLMs can influence decisions in areas such as medical classifications and educational hiring. |
| Approach: | They conducted two interactive experiments on partisan bias in large language models while completing tasks with either a biased liberal, biased conservative, or unbiased control model. |
| Outcome: | The results show that prior knowledge of AI is weakly correlated with a reduction of the bias, suggesting that AI education can be crucial for mitigating bias effects. |
Large Language Models Help Humans Verify Truthfulness – Except When They Are Convincingly Wrong (2024.naacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly used for accessing information on the web. |
| Approach: | They conduct experiments with 80 crowdworkers to compare LLMs with search engines . they ask LLM to provide contrastive information to reduce over-reliance on LLM . |
| Outcome: | The results show that LLMs can outperform search engines but not LLM explanations . the study shows that LMS explanations are not reliable replacements for reading retrieved passages compared to search engines alone. |
Beyond Performance: Quantifying and Mitigating Label Bias in LLMs (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models exhibit undesirable preference toward predicting certain answers over others, despite their adaptability to diverse tasks. |
| Approach: | They propose a label bias calibration method that outperforms recent calibration approaches for improving performance and mitigating label bias. |
| Outcome: | The proposed method outperforms calibration approaches for improving performance and mitigating label bias. |
Conformity in Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Conformity is a form of social influence that affects the way people respond to information. |
| Approach: | They adapt psychological experiments to examine the extent of conformity in large language models. |
| Outcome: | The proposed interventions mitigate conformity by reducing the naturalness of majority tones and reducing instruction-tuned models. |
Human Alignment: How Much Do We Adapt to LLMs? (2025.acl-short)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are becoming a common part of our lives, yet few studies have examined how they influence our behavior. |
| Approach: | They propose a cooperative language game in which players aim to converge on a word and play a game in a group. |
| Outcome: | The proposed game shows that humans notice and adapt to differences regardless of whether they are aware they are interacting with an LLM. |
How Does Cognitive Bias Affect Large Language Models? A Case Study on the Anchoring Effect in Price Negotiation Simulations (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Cognitive biases can be observed in LLMs, affecting their reliability in real-world applications. |
| Approach: | They investigate the anchoring effect in LLM-driven price negotiations . reasoning models are less prone to the anchor effect, they find . |
| Outcome: | The proposed study shows that LLMs are influenced by the anchoring effect like humans . reasoning models are less prone to the anchor effect, but personality traits are not affected . |
Bias Beware: The Impact of Cognitive Biases on LLM-Driven Product Recommendations (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have revolutionized product recommenders, but their susceptibility to adversarial manipulations is difficult to detect. |
| Approach: | They propose to use large language models to investigate cognitive biases as adversarial strategies in product research using LLMs. |
| Outcome: | The proposed approach is the first to tap into human psychological principles, making such manipulations hard to detect. |
Systematic Biases in LLM Simulations of Debates (2024.emnlp-main)
Copied to clipboard
| Challenge: | Current research suggests that LLM-based agents become increasingly human-like in their performance, sparking interest in using these AI agents as substitutes for human participants in behavioral studies. |
| Approach: | They propose to use LLMs to simulate political debates on topics that are important aspects of people’s day-to-day lives and decision-making processes. |
| Outcome: | The proposed model can simulate political debates on topics that are important aspects of people’s day-to-day lives and decision-making processes. |
Do LLMs Align Human Values Regarding Social Biases? Judging and Explaining Social Biases with LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models can lead to undesired consequences when misaligned with human values . previous studies have shown misalignment of LLMs with human value using expert-designed or agent-based emulated bias scenarios . |
| Approach: | They investigate whether large language models (LLMs) are misaligned with human values . they find no significant differences in understanding of HVSB between LLMs . |
| Outcome: | The results show that large language models do not have lower misalignment rates and attack success rates . the study also shows that smaller language models have the ability to explain HVSB . |
Factuality of Large Language Models: A Survey (2024.emnlp-main)
Copied to clipboard
Yuxia Wang, Minghan Wang, Muhammad Arslan Manzoor, Fei Liu, Georgi Georgiev, Rocktim Das, Preslav Nakov
| Challenge: | Large language models (LLMs) are factually incorrect, which limits their applicability in real-world scenarios. |
| Approach: | They analyze existing work to identify major challenges and their associated causes . they propose to evaluate LLMs using a variety of measures to mitigate factual errors . |
| Outcome: | The proposed methods are based on a variety of datasets and proposed strategies to mitigate factual errors. |