HumT DumT: Measuring and controlling human-like language in LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Human-like language might improve user experience, but might also lead to deception, overreliance, and stereotyping. |
| Approach: | They introduce HumT and SocioT, metrics for human-like tone in LLM outputs . HumT measures human-type tone and other dimensions of social perceptions in text data . |
| Outcome: | The proposed method reduces human-like tone while preserving model performance. |
Similar Papers
Can You Make It Sound Like You? Post-Editing LLM-Generated Text for Personal Style (2026.acl-long)
Copied to clipboard
| Challenge: | Despite the growing use of large language models for writing tasks, it remains unclear whether users can effectively reshape LLM-generated text to reflect their personal style. |
| Approach: | They conduct an online study in which participants post-edit LLM-generated drafts for writing tasks where personal style matters to them. |
| Outcome: | The results show that post-editing increases stylistic similarity to unassisted writing and reduces similarity with fully LLM-generated output. |
Bias Mitigation or Cultural Commonsense? Evaluating LLMs with a Japanese Dataset (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have demonstrated that large language models exhibit social biases . however, debiasing methods may degrade the capabilities of LLMs if they are not properly evaluated . |
| Approach: | They propose a Japanese benchmark to evaluate social biases and cultural commonsense in large language models in a unified format. |
| Outcome: | The proposed method degrades the performance of the LLMs on the cultural commonsense task by 75%. |
Dissecting Human and LLM Preferences (2024.acl-long)
Copied to clipboard
| Challenge: | a recent study shows that human and Large Language Model preferences are important for model fine-tuning and evaluation. |
| Approach: | They dissect the preferences of human and 32 different Large Language Models to understand their quantitative composition. |
| Outcome: | The proposed model is compared with 32 different large language models using real-world user-model conversations. |
Linguistic and Embedding-Based Profiling of Texts Generated by Humans and Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have focused on using LLMs to classify text as either human-written or machine-generated . |
| Approach: | They characterize human-written and machine-generated texts using a set of linguistic features across different linguistic levels such as morphology, syntax, and semantics. |
| Outcome: | The proposed model reveals that human-written texts exhibit simpler syntactic structures and more diverse semantic content. |
Real or Robotic? Assessing Whether LLMs Accurately Simulate Qualities of Human Responses in Human-LLM Dialogue (2026.findings-acl)
Copied to clipboard
Jonathan Ivey, Shivani Kumar, Jiayu Liu, Hua Shen, Sushrita Rakshit, Rohan Raju, Haotian Zhang, Aparna Ananthasubramaniam, Junghwan Kim, Bowen Yi, Dustin Wright, Abraham Israeli, Anders Giovanni Møller, Lechen Zhang, David Jurgens
| Challenge: | Recent work has sought to use large language models to simulate human-human and human-LLM interactions. |
| Approach: | They use a large-scale dataset to generate a paired LLM-LLM and human-LLm dialogues from the WildChat dataset and quantify how well they align with their human counterparts. |
| Outcome: | The proposed models perform similarly in simulating English, Chinese, and Russian dialogues. |
Decoding LLM Personality Measurement: Forced-Choice vs. Likert (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent research has focused on investigating the psychological characteristics of Large Language Models (LLMs), emphasizing the importance of comprehending their behavioral traits. |
| Approach: | They evaluated six Large Language Models: Llama-3.1-8B, GLM-4-9B, Claude-3.5-sonnet, and Deepseek-V3 and used the forced-choice test to assess their personality traits. |
| Outcome: | The forced-choice test is more reliable and more accurate than the likert scale and forced-CHOICE test results for LLMs' Big Five personality scores. |
Modeling Human Subjectivity in LLMs Using Explicit and Implicit Human Factors in Personas (2024.findings-emnlp)
Copied to clipboard
Salvatore Giorgi, Tingting Liu, Ankit Aich, Kelsey Isman, Garrick Sherman, Zachary Fried, João Sedoc, Lyle Ungar, Brenda Curtis
| Challenge: | Large language models (LLMs) are increasingly being used in human-centered social scientific tasks, such as data annotation, synthetic data creation, and engaging in dialog. |
| Approach: | They propose to prompt LLMs with human-like personas and ask them to answer as if they were a specific human, either explicitly, with exact demographics, political beliefs, and lived experiences, or implicitly via names prevalent in specific populations. |
| Outcome: | The proposed model is based on explicit, explicit, and implicit personas, and fails to show implicit biases. |
Exploring the Impact of Personality Traits on LLM Toxicity and Bias (2025.emnlp-main)
Copied to clipboard
| Challenge: | anthropomorphic LLMs are being developed to serve diversified roles, but content safety concerns remain regarding their toxicity and toxicity. |
| Approach: | They propose to assign personality traits to large language models (LLMs) to reduce toxic language and social biases in their outputs by using the widely accepted HEXACO personality framework developed in social psychology. |
| Outcome: | The proposed model is able to perform on three toxic and bias benchmarks and shows that assigning personality traits reduces bias and toxicity similar to humans’ correlations between personality traits and toxic behaviors. |
Your Mileage May Vary: How Empathy and Demographics Shape Human Preferences in LLM Responses (2025.findings-emnlp)
Copied to clipboard
| Challenge: | large language models (LLMs) increasingly assist subjective decision-making . prior work uses aggregate human judgments, but demographic variation and its linguistic drivers remain underexplored. |
| Approach: | They analyze how demographic background and empathy level correlate with LLM-generated dilemma responses . they also identify markers that predict group-level differences . |
| Outcome: | The authors show that demographic background and empathy level correlate with LLM preferences . their findings highlight the need for demographically informed LLM evaluations. |
On the Reliability of Psychological Scales on Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent research has focused on examining Large Language Models’ characteristics from a psychological standpoint, acknowledging the necessity of understanding their behavioral characteristics. |
| Approach: | They propose to examine the reliability of personality tests to LLMs by using psychological scales. |
| Outcome: | The proposed model can represent diverse personalities with specific prompt instructions. |