Comparing human and LLM politeness strategies in free production (2025.emnlp-main)
Copied to clipboard
| Challenge: | Polite speech poses a fundamental alignment challenge for large language models (LLMs). |
| Approach: | They compare human and LLM responses to English-language scenarios to determine whether they employ a similarly context-sensitive repertoire. |
| Outcome: | The results show that large models replicate key effects from the computational pragmatics literature and human evaluators prefer LLM-generated responses in open-ended contexts. |
Similar Papers
Dissecting Human and LLM Preferences (2024.acl-long)
Copied to clipboard
| Challenge: | a recent study shows that human and Large Language Model preferences are important for model fine-tuning and evaluation. |
| Approach: | They dissect the preferences of human and 32 different Large Language Models to understand their quantitative composition. |
| Outcome: | The proposed model is compared with 32 different large language models using real-world user-model conversations. |
Human Alignment: How Much Do We Adapt to LLMs? (2025.acl-short)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are becoming a common part of our lives, yet few studies have examined how they influence our behavior. |
| Approach: | They propose a cooperative language game in which players aim to converge on a word and play a game in a group. |
| Outcome: | The proposed game shows that humans notice and adapt to differences regardless of whether they are aware they are interacting with an LLM. |
How Hypocritical Is Your LLM judge? Listener-Speaker Asymmetries in the Pragmatic Competence of Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly studied as repositories of linguistic knowledge. |
| Approach: | They compare LLMs’ performance as pragmatic listeners and as pragmatic speakers . they find a robust asymmetry between pragmatic evaluation and pragmatic generation . |
| Outcome: | The proposed models perform better as listeners than speakers, and produce more appropriate language than speakers. |
Systematic Biases in LLM Simulations of Debates (2024.emnlp-main)
Copied to clipboard
| Challenge: | Current research suggests that LLM-based agents become increasingly human-like in their performance, sparking interest in using these AI agents as substitutes for human participants in behavioral studies. |
| Approach: | They propose to use LLMs to simulate political debates on topics that are important aspects of people’s day-to-day lives and decision-making processes. |
| Outcome: | The proposed model can simulate political debates on topics that are important aspects of people’s day-to-day lives and decision-making processes. |
Do LLMs Align Human Values Regarding Social Biases? Judging and Explaining Social Biases with LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models can lead to undesired consequences when misaligned with human values . previous studies have shown misalignment of LLMs with human value using expert-designed or agent-based emulated bias scenarios . |
| Approach: | They investigate whether large language models (LLMs) are misaligned with human values . they find no significant differences in understanding of HVSB between LLMs . |
| Outcome: | The results show that large language models do not have lower misalignment rates and attack success rates . the study also shows that smaller language models have the ability to explain HVSB . |
Real or Robotic? Assessing Whether LLMs Accurately Simulate Qualities of Human Responses in Human-LLM Dialogue (2026.findings-acl)
Copied to clipboard
Jonathan Ivey, Shivani Kumar, Jiayu Liu, Hua Shen, Sushrita Rakshit, Rohan Raju, Haotian Zhang, Aparna Ananthasubramaniam, Junghwan Kim, Bowen Yi, Dustin Wright, Abraham Israeli, Anders Giovanni Møller, Lechen Zhang, David Jurgens
| Challenge: | Recent work has sought to use large language models to simulate human-human and human-LLM interactions. |
| Approach: | They use a large-scale dataset to generate a paired LLM-LLM and human-LLm dialogues from the WildChat dataset and quantify how well they align with their human counterparts. |
| Outcome: | The proposed models perform similarly in simulating English, Chinese, and Russian dialogues. |
Investigating Human and LLMs’ Decisions in Unverifiable Environments: A Case Study with GitHub Activity Overview (2026.findings-acl)
Copied to clipboard
| Challenge: | examining the behaviors of Large Language Models as artificial social actors is underexplored, especially in unverifiable scenarios where conventional benchmarking has little to help improve their abilities. |
| Approach: | They propose a method to collect, compare, and reason about human and LLMs' decisions in an unverifiable scenario and use it to examine their behaviors. |
| Outcome: | The proposed method compared human and LLM decisions in an unverifiable scenario on GitHub and found that proprietary LLMs behave more like humans than open-source LLM systems. |
Constructing Your Model’s Value Distinction: Towards LLM Alignment with Anchor Words Tuning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a study of large language models (LLMs) shows that they can generate outputs that are honest, positive, harmless, etc. |
| Approach: | They propose a method that amplifies logits difference between positive and negative tokens . they propose to use the logits gap to generate positive and positive tokens after alignment . |
| Outcome: | The proposed method achieves effective alignment, but requires fewer computational resources compared to training-time alignment methods. |
A fine-grained comparison of pragmatic language understanding in humans and language models (2023.acl-long)
Copied to clipboard
| Challenge: | Pragmatics and non-literal language understanding are essential to human communication . a long-standing challenge for artificial language models is to capture pragmatics . |
| Approach: | They compare language models and humans on seven pragmatic phenomena using curated English materials. |
| Outcome: | The proposed model achieves high accuracy and matches human error patterns . the results suggest pragmatic behaviors can emerge in models without explicit representations of mental states . |
Comparing Bad Apples to Good Oranges Aligning Large Language Models via Joint Preference Optimization (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have shown that acquiring human preferences by comparing generations is not effective for large language models. |
| Approach: | They propose a preference optimization objective that elicits preferences jointly over the instruction-response pairs. |
| Outcome: | The proposed approach outperforms prior preference optimizations by 5.2% and 3.3% in summarization and open-ended dialogue datasets. |