| Challenge: | Language models (LMs) are used in decision-making systems and as interactive assistants. |
| Approach: | They propose to prompt 11 LMs on rules-of-thumb and compare their outputs with 100 human annotators. |
| Outcome: | The proposed model is compared with 100 human annotators to find out if they are inclusive of diverse human values. |
Similar Papers
Do LLMs Align Human Values Regarding Social Biases? Judging and Explaining Social Biases with LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models can lead to undesired consequences when misaligned with human values . previous studies have shown misalignment of LLMs with human value using expert-designed or agent-based emulated bias scenarios . |
| Approach: | They investigate whether large language models (LLMs) are misaligned with human values . they find no significant differences in understanding of HVSB between LLMs . |
| Outcome: | The results show that large language models do not have lower misalignment rates and attack success rates . the study also shows that smaller language models have the ability to explain HVSB . |
Whose Emotions and Moral Sentiments do Language Models Reflect? (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing research has focused on positional alignment, which measures how closely the models mimic the opinions and stances of different social groups. |
| Approach: | They define the problem of affective alignment, which measures how LMs’ emotional and moral tone represents those of different groups. |
| Outcome: | The results show that the models represent the perspectives of some social groups better than others, suggesting a systemic bias within LMs. |
Can Language Models Reason about Individualistic Human Values and Preferences? (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods and evaluation frameworks for achieving pluralistic alignment are limited by the diversity of people, which is pre-specified and coarsely categorized, papering over individuality. |
| Approach: | They propose to use a dataset transformed from the influential World Values Survey to study language models on the specific challenge of individualistic value reasoning. |
| Outcome: | The proposed model can predict individualistic values with accuracies between 55% and 65%, while a precise description of individualistic value judgments cannot be approximated only via demographic information. |
Knowledge of cultural moral norms in large language models (2023.acl-long)
Copied to clipboard
| Challenge: | Existing studies do not examine moral variation in a diverse cultural setting. |
| Approach: | They investigate whether monolingual English language models capture moral variation across cultures . they use data from the World Values Survey and PEW global surveys . |
| Outcome: | The proposed models predict moral norms worse than the English models reported previously . the models improve inference across countries at the expense of an accurate estimate . |
Your Mileage May Vary: How Empathy and Demographics Shape Human Preferences in LLM Responses (2025.findings-emnlp)
Copied to clipboard
| Challenge: | large language models (LLMs) increasingly assist subjective decision-making . prior work uses aggregate human judgments, but demographic variation and its linguistic drivers remain underexplored. |
| Approach: | They analyze how demographic background and empathy level correlate with LLM-generated dilemma responses . they also identify markers that predict group-level differences . |
| Outcome: | The authors show that demographic background and empathy level correlate with LLM preferences . their findings highlight the need for demographically informed LLM evaluations. |
Missing the Margins: A Systematic Literature Review on the Demographic Representativeness of LLMs (2025.findings-acl)
Copied to clipboard
| Challenge: | 211 studies on the demographic representativeness of large language models have conflicting results . 29% of the studies report positive conclusions on the representativeness, 30% do not evaluate LLMs across multiple demographic categories or within demographic subcategories. |
| Approach: | 211 papers review the representativeness of large language models . authors recommend more precise evaluation methods and comprehensive documentation of demographic attributes . |
| Outcome: | 211 studies on the representativeness of large language models are reviewed . 29% of the studies report positive conclusions, but 30% fail to specify subcategories . authors recommend more precise evaluation methods and documentation of demographic attributes . |
NormAd: A Framework for Measuring the Cultural Adaptability of Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are widely used and engage millions of users from diverse contexts and cultures. |
| Approach: | They propose an evaluation framework to assess LLMs’ cultural adaptability by measuring their ability to judge social acceptability across varying levels of cultural norm specificity. |
| Outcome: | The proposed model shows stronger adaptability to English-centric cultures over those from the Global South. |
Aligning Language Models to User Opinions (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Personality is a defining feature of human beings, shaped by a complex interplay of demographic characteristics, moral principles, and social experiences. |
| Approach: | They use public opinion surveys to model past user opinions in addition to user demographics and ideology to achieve up to 7 points accuracy gains in predicting public opinions from survey questions. |
| Outcome: | The proposed model achieves 7 points accuracy gains in predicting public opinions from public opinion surveys across a broad set of topics. |
Can Persona-Prompted LLMs Emulate Subgroup Values? An Empirical Analysis of Generalisability and Fairness in Cultural Alignment (2026.acl-long)
Copied to clipboard
Bryan Chen Zhengyu Tan, Zhengyuan Liu, Xiaoyuan Yi, Jing Yao, Xing Xie, Nancy F. Chen, Roy Ka-Wei Lee
| Challenge: | Current alignment paradigms treat "human values" as a monolithic entity, ignoring the fact that many societies are a mosaic of diverse subgroups with distinct and sometimes conflicting values, preferences, and norms. |
| Approach: | They examine whether Large Language Models can emulate distinct cultural values of subgroups . they use a global value survey to examine the value landscape of a multicultural society . |
| Outcome: | The proposed model improves on unseen, out-of-distribution subgroups by 17.4% . the model widens the disparity between subgroup groups when measured by distance-aware metrics. |
QueerGen: How LLMs Reflect Societal Norms on Gender and Sexuality in Sentence Completion Task (2026.findings-eacl)
Copied to clipboard
| Challenge: | Autoregressive Language Models (ARLMs) partially mitigate these patterns, while closed-access ARLMs tend to produce more harmful outputs for unmarked subjects. |
| Approach: | They examine whether explicit information about a subject’s gender or sexuality influences LLM responses across three subject categories: queer-marked, non-queer-mark, and the normalized "unmarked" category. |
| Outcome: | The proposed models reproduce normative social assumptions, but the form and degree of bias depend on model characteristics, which may redistribute—but not eliminate—representational harms. |