Roleplay-doh: Enabling Domain-Experts to Create LLM-simulated Patients via Eliciting and Adhering to Principles (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for improving LLMs in simulations are limited due to privacy concerns and limited domain knowledge. |
| Approach: | They propose a pipeline that elicits qualitative feedback from a domain-expert and transforms it into a set of principles that govern an LLM-prompted roleplay. |
| Outcome: | The proposed pipeline shows a 30% improvement in response quality and principle following for the downstream task. |
Similar Papers
HER: Human-like Reasoning and Reinforcement Learning for LLM Role-playing (2026.findings-acl)
Copied to clipboard
Chengyu Du, Xintao Wang, Aili Chen, Weiyuan Li, Rui Xu, Junteng Liu, Zishan Huang, Rong Tian, Zijun Sun, Yuhao Li, Liheng Feng, Deming Ding, Pengyu Zhao, Yanghua Xiao
| Challenge: | Existing models for LLM role-playing lack high-quality datasets with explicit reasoning traces and reliable reward signals aligned with human preferences. |
| Approach: | They propose a unified framework for cognitive-level persona simulation that strictly distinguishes characters’ first-person thinking processes from LLMs’ third-person reasoning. |
| Outcome: | The proposed framework outperforms the Qwen3-32B baseline model and achieves a 30.26% and 14.97% performance on the minimax benchmarks. |
Memory-Driven Role-Playing: Evaluation and Enhancement of Persona Knowledge Utilization in LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing models fail to recall and accurately apply designated persona knowledge without explicit cues . memory-driven role-playing paradigms are attracting significant interest . |
| Approach: | They propose a memory-driven role-playing paradigm that frames persona knowledge as the LLM's internal memory store and a prompting architecture that guides structured memory retrieval and response generation. |
| Outcome: | The proposed paradigm provides a comprehensive diagnostic for four-stage role-playing abilities across 12 LLMs. |
RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models (2024.findings-acl)
Copied to clipboard
Noah Wang, Z.y. Peng, Haoran Que, Jiaheng Liu, Wangchunshu Zhou, Yuhan Wu, Hongcheng Guo, Ruitong Gan, Zehao Ni, Jian Yang, Man Zhang, Zhaoxiang Zhang, Wanli Ouyang, Ke Xu, Wenhao Huang, Jie Fu, Junran Peng
| Challenge: | Large Language Models (LLMs) have paved the way for complex tasks such as role-playing. |
| Approach: | They propose a framework to benchmark, elicit, and enhance role-playing abilities in Large Language Models. |
| Outcome: | The proposed framework improves role-playing abilities with 168,093 samples. |
Can Role Vectors Affect LLM Behaviour? (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study has shown that personas influence LLM performance, but their direct impact remains unclear. |
| Approach: | They propose a novel approach to guiding LLM behaviour through role vectors . they construct 29 role vector derived from model activations and evaluate their impact . |
| Outcome: | The proposed approach improves in-domain task performance while yielding unexpected gains. |
Eeyore: Realistic Depression Simulation via Expert-in-the-Loop Supervised and Preference Optimization (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have been explored for mental healthcare training and therapy client simulation, but they fail to authentically capture diverse client traits and psychological conditions. |
| Approach: | They propose an 8B model optimized for realistic depression simulation with expert input at every stage. |
| Outcome: | The model outperforms GPT-4o in linguistic authenticity and profile adherence. |
Enhancing Persona Consistency for LLMs’ Role-Playing using Persona-Aware Contrastive Learning (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for analyzing and analyzing large language models (LLMs) lack of emotion and fine-grained role awareness limits the model’s ability to provide personalized and diverse interactions further. |
| Approach: | They propose an annotation-free framework to align LLMs’ behavior during role-playing, enhancing the model’s role consistency. |
| Outcome: | The proposed framework outperforms vanilla LLMs under automatic evaluation methods and human expert evaluation. |
Knowing-but-Doing: Diagnosing and Defending Role-Play-Driven LLMs Jailbreaks via Moral Disengagement (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly used in role-play scenarios, but their safety implications remain under-characterized. |
| Approach: | They propose a diagnostic benchmark for role-play jailbreaks based on Bandura’s Moral Disengagement theory and propose 'MD-Trace' based defense that reduces attack success while maintaining Role Fidelity. |
| Outcome: | The proposed framework improves safety behavior for benign personas while increasing unsafe compliance for malicious ones. |
LLM-based Medical Assistant Personalization with Short- and Long-Term Memory Coordination (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing studies have focused on learning and enhancing large language models to understand and generate natural language. |
| Approach: | They propose a computational bionic memory mechanism equipped with a parameter-efficient fine-tuning schema to personalize medical assistants. |
| Outcome: | The proposed method can enhance the response with aware of previous mistakes for new queries during a dialogue session, but the training costs are prohibitive. |
Can AI Relate: Testing Large Language Model Response for Mental Health Support (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) are already being piloted for clinical use in hospitals . recent failures of the Tessa chatbot have led to doubts about their reliability in high-stakes settings. |
| Approach: | They propose safety guidelines for the potential deployment of large language models for mental health response. |
| Outcome: | The proposed framework measures equity in empathy and adherence of LLM responses to motivational interviewing theory. |
LLMs Can Simulate Standardized Patients via Agent Coevolution (2025.acl-long)
Copied to clipboard
Zhuoyun Du, LujieZheng LujieZheng, Renjun Hu, Yuyang Xu, Xiawei Li, Ying Sun, Wei Chen, Jian Wu, Haolei Cai, Haochao Ying
| Challenge: | Training medical personnel using standardized patients (SPs) remains a complex challenge, necessitating extensive domain expertise and role-specific practice. |
| Approach: | They propose a simulated patient framework that allows patient agents to simulate diagnostic process through multi-turn dialogues. |
| Outcome: | The proposed framework improves over existing reasoning methods by more than 10% in requirement alignment and better human preference after evolving over 200 cases for 10 hours with excellent generalizability. |