The Greatest Good Benchmark: Measuring LLMs’ Alignment with Utilitarian Moral Dilemmas (2024.emnlp-main)
Copied to clipboard
| Challenge: | Our analysis across 15 diverse LLMs reveals consistently encoded moral preferences that diverge from established moral theories and lay population moral standards. |
| Approach: | They propose to evaluate the moral judgments of large language models using utilitarian dilemmas to determine their moral alignment. |
| Outcome: | The findings highlight the ‘artificial moral compass’ of Large Language Models, offering insights into their moral alignment. |
Similar Papers
The Pluralistic Moral Gap: Understanding Moral Judgment and Value Differences between Humans and Large Language Models (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing studies have shown that Large Language Models (LLMs) are not fully aligned with human moral judgments. |
| Approach: | They propose a dataset of 1,618 real-world moral dilemmas paired with a distribution of human moral judgments consisting of a binary evaluation and a free-text rationale to examine how closely LLMs align with human moral judgements. |
| Outcome: | The proposed model reproduces human judgments only under high consensus; alignment deteriorates sharply when human disagreement increases. |
Ethical Reasoning over Moral Alignment: A Case and Framework for In-Context Ethical Policies in LLMs (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a paper by a team of researchers proposes that large language models should be morally aligned to ethical principles . a moral compass is a model that integrates moral dilemmas with moral principles pertaining to different foramlisms of normative ethics . |
| Approach: | They propose to infuse generic ethical reasoning capabilities into large-scale models . they argue that LLMs should take a moral stance on value pluralism . |
| Outcome: | a new ethical reasoning framework integrates moral dilemmas with moral principles . the framework is based on the results of a hypothetical case study on a large-scale model . |
Value Compass Benchmarks: A Comprehensive, Generative and Self-Evolving Platform for LLMs’ Value Evaluation (2025.acl-demo)
Copied to clipboard
Jing Yao, Xiaoyuan Yi, Shitong Duan, Jindong Wang, Yuzhuo Bai, Muhua Huang, Yang Ou, Scarlett Li, Peng Zhang, Tun Lu, Zhicheng Dou, Maosong Sun, James Evans, Xing Xie
| Challenge: | Current evaluation methods for large language models face two key challenges: 1. evaluation validity and 2. Result interpretation reduce the pluralistic and incommensurable values to one-dimensional scores. |
| Approach: | They propose a platform for comprehensive value diagnosis of large language models (LLMs) that provides a generative evaluation paradigm that automatically creates real-world test items co-evolving with ever-advancing LLMs. |
| Outcome: | The proposed platform provides a framework for comprehensive value diagnosis of large language models (LLMs) with fine-grained scores and case studies across 27 value dimensions for 33 leading LLMs, customized comparisons, and visualized analysis of LLM’s alignment with cultural values. |
Beyond Value Benchmarks: Measuring Value-Structure Alignment in Large Language Models via Symmetric Q-Sorts (2026.acl-long)
Copied to clipboard
| Challenge: | Existing evaluations of Large Language Models (LLMs) focus on item-level behavioral metrics without capturing how models prioritize competing values as a whole. |
| Approach: | They propose a symmetric human-LLM evaluation framework to measure value-structure alignment . they evaluate 12 LLMs across four model families via 240 replicated Q-sorts . |
| Outcome: | The proposed framework measures value-structure alignment across four model families. |
Value Portrait: Assessing Language Models’ Values through Psychometrically and Ecologically Valid Items (2025.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks rely on human annotations that are vulnerable to value-related biases. |
| Approach: | They propose a value portrait benchmark that uses items that capture real-life user-LLM interactions and a rated item based on its similarity to their own thoughts to determine reliability. |
| Outcome: | The proposed framework improves the relevance of assessment results to real-world LLM usage by allowing human subjects to rate items with similarity to their own thoughts and derived correlations between these ratings and the subjects’ actual value scores. |
Unintended Harms of Value-Aligned LLMs: Psychological and Empirical Insights (2025.acl-long)
Copied to clipboard
| Challenge: | Value-aligned LLMs are more prone to harmful behavior than fine-tuned models . value-aligned models generate text according to the aligned values, which can amplify harmful outcomes. |
| Approach: | They propose to use in-context alignment methods to enhance the safety of value-aligned LLMs. |
| Outcome: | The proposed methods improve value alignment and safety, the authors say . value-aligned models are more prone to harmful behavior than fine-tuned models . |
Flames: Benchmarking Value Alignment of LLMs in Chinese (2024.naacl-long)
Copied to clipboard
Kexin Huang, Xiangyang Liu, Qianyu Guo, Tianxiang Sun, Jiawei Sun, Yaru Wang, Zeyang Zhou, Yixu Wang, Yan Teng, Xipeng Qiu, Yingchun Wang, Dahua Lin
| Challenge: | Existing benchmarks for large language models (LLMs) do not accurately uncover safety vulnerabilities in LLMs. |
| Approach: | They propose a value alignment benchmark called Flames that encompasses both harmlessness principles and a unique morality dimension that integrates specific Chinese values such as harmony. |
| Outcome: | The proposed model performs poorly on Flames, particularly in safety and fairness dimensions. |
A Survey on Personalized Alignment—The Missing Piece for Large Language Models in Real-World Applications (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable capabilities, yet their transition to real-world applications reveals a critical limitation: the inability to adapt to individual preferences while maintaining alignment with universal human values. |
| Approach: | They propose a framework that enables LLMs to adapt their behavior within ethical boundaries based on individual preferences. |
| Outcome: | The proposed framework analyzes implementation approaches and evaluates their effectiveness across various scenarios. |
The Staircase of Ethics: Probing LLM Value Priorities through Multi-Step Induction to Complex Moral Dilemmas (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluations of LLMs' moral reasoning capabilities rely on single-step evaluations, ignoring how models adapt to evolving ethical challenges. |
| Approach: | They propose a framework to evaluate evolving moral judgments of large language models (LLMs) using multi-step moral dilemma questionnaires. |
| Outcome: | The proposed framework enables a fine-grained analysis of how LLMs adjust their moral reasoning across escalating dilemmas. |
Re-evaluating Automatic LLM System Ranking for Alignment with Human Preference (2025.findings-naacl)
Copied to clipboard
| Challenge: | Evaluating and ranking the capabilities of different LLMs is crucial for understanding their performance and alignment with human preferences. |
| Approach: | They propose a system-level evaluation framework that ranks LLMs based on their alignment with human preferences. |
| Outcome: | The proposed framework aims to rank LLMs based on their performance and alignment with human preferences. |