Who Holds the Pen? Caricature and Perspective in LLM Retellings of History (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models are increasingly used to simulate human perspectives, authors say . authors: asymmetries in tone, stance, and emphasis can quietly, yet systematically, distort how history is told and remembered. |
| Approach: | They analyze LLM-generated responses across 197 historically significant events . they find that LLMs reliably distinguish persona-based responses from neutral baselines . |
| Outcome: | The findings show that LLMs distinguish persona-based responses from neutral baselines and that directly affected personas exhibit higher exaggeration. |
Similar Papers
CoMPosT: Characterizing and Evaluating Caricature in LLM Simulations (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent work has aimed to capture nuances of human behavior by using LLMs to simulate responses from demographics in social science experiments and public opinion surveys. |
| Approach: | They propose a framework to characterize LLM simulations using four dimensions: Context, Model, Persona, and Topic. |
| Outcome: | The proposed framework measures open-ended LLM simulations’ susceptibility to caricature, defined via two criteria: individuation and exaggeration. |
Systematic Biases in LLM Simulations of Debates (2024.emnlp-main)
Copied to clipboard
| Challenge: | Current research suggests that LLM-based agents become increasingly human-like in their performance, sparking interest in using these AI agents as substitutes for human participants in behavioral studies. |
| Approach: | They propose to use LLMs to simulate political debates on topics that are important aspects of people’s day-to-day lives and decision-making processes. |
| Outcome: | The proposed model can simulate political debates on topics that are important aspects of people’s day-to-day lives and decision-making processes. |
Bias in the Mirror : Are LLMs opinions robust to their own adversarial attacks (2025.acl-long)
Copied to clipboard
| Challenge: | Existing work on large language models lacks robustness, highlighting the limitations of such models. |
| Approach: | They propose a novel approach where two LLMs engage in self-debate to persuade a neutral version of the model. |
| Outcome: | The proposed approach examines whether large language models are robust during interactions and whether they are susceptible to reinforcing misinformation or shifting to harmful viewpoints. |
LLM Tropes: Revealing Fine-Grained Values and Opinions in Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to evaluate latent values and opinions in large language models suffer from three notable shortcomings. |
| Approach: | They propose to analyze 156k LLM responses to 62 propositions of the Political Compass Test (PCT) generated by 6 LLMs using 420 prompt variations. |
| Outcome: | The proposed analysis of 156k LLM responses to the Political Compass Test (PCT) generated by 6 LLMs shows that tropes are recurrent and consistent across prompts. |
How LLMs Comprehend Temporal Meaning in Narratives: A Case Study in Cognitive Evaluation of LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models exhibit increasingly sophisticated linguistic capabilities, yet the extent to which these models reflect human-like cognition versus advanced pattern recognition remains an open question. |
| Approach: | They conduct a series of targeted experiments to assess whether LLMs construct semantic representations and pragmatic inferences in a human-like manner. |
| Outcome: | The proposed framework can be used to assess the cognitive and linguistic capabilities of large language models (LLMs). |
Evaluating Large Language Model Biases in Persona-Steered Generation (2024.findings-acl)
Copied to clipboard
| Challenge: | a recent wave of powerful new large language models has raised concerns that their expressed opinions may be biased towards certain political, national or moral viewpoints. |
| Approach: | They define an incongruous persona as a persona with multiple traits where one trait makes its other traits less likely in human survey data. |
| Outcome: | The results show that LLMs are less steerable towards incongruous personas than congruous ones . the models that are fine-tuned with RLHF are more steerable, especially towards stances associated with political liberals and women . |
An Empirical Analysis of the Writing Styles of Persona-Assigned LLMs (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent efforts to "personalize" large language models by assigning them specific personas are limited by current knowledge of how well they perform. |
| Approach: | They use a style embedding model to analyze writing styles of persona-assigned LLMs . they find significant style differences between personas using Kullback-Leibler divergence . |
| Outcome: | The proposed model shows significant differences in writing styles among personas across socio-demographic groups. |
Analysing LLM Persona Generation and Fairness Interpretation in Polarised Geopolitical Contexts (2026.eacl-srw)
Copied to clipboard
| Challenge: | Large language models are increasingly used for social simulation and persona generation. |
| Approach: | They analysed personas generated for Palestinian and Israeli identities by popular LLMs across 640 experimental conditions, varying context and assigned roles. |
| Outcome: | The results show that large language models are increasingly utilised for social simulation and persona generation. |
Fine-Tuned LLMs are “Time Capsules” for Tracking Societal Bias Through Books (2025.naacl-long)
Copied to clipboard
| Challenge: | We develop a corpus comprising 593 fictional books across seven decades (1950-2019) to track bias evolution. |
| Approach: | They develop a method to trace and quantify bias evolution using fine-tuned LLMs on fictional books across seven decades to track bias evolution. |
| Outcome: | The proposed method traces and quantifies bias evolution in a corpus of 593 fictional books across seven decades. |
From Fallback to Frontline: When Can LLMs be Superior Annotators of Human Perspectives? (2026.findings-acl)
Copied to clipboard
| Challenge: | large language models are often used as annotators at scale, but are not faithful estimators of human perspectives. |
| Approach: | They characterize the conditions under which large language models outperform human annotators . they find they are statistically superior frontline estimators based on low variance . |
| Outcome: | The proposed model outperforms human annotators when predicting subgroup opinions on subjective tasks. |