PsychEval: A Multi-Session and Multi-Therapy Benchmark for High-Realism AI Psychological Counselor (2026.findings-acl)
Copied to clipboard
Qianjun Pan, Junyi Wang, Jie Zhou, Yutao Yang, Junsong Li, Kaiyin Xu, Yougen Zhou, Yihan Li, JingYuan Zhao, Qin Chen, Ningning Zhou, Kai Chen, Liang He
| Challenge: | Existing models focus on a single therapy, but complex cases require flexible strategies among various therapies. |
| Approach: | They propose a multi-session, multi-therapy, and highly realistic benchmark . it is designed to address three key challenges: 1) can we train a highly realistic AI counselor? 2) How to systematically evaluate an AI counselor?" |
| Outcome: | The proposed benchmark is annotated with extensive professional skills and includes over 677 meta-skills and 4577 atomic skills. |
Similar Papers
Psychological Counseling Cannot Be Achieved Overnight: Automated Psychological Counseling Through Multi-Session Conversations (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on single-session counseling are limited to a single-session setting. |
| Approach: | They propose to use a large language model to deliver automated psychological counseling to a dataset constructed using real client profiles from publicly available psychological case reports. |
| Outcome: | The proposed model performs better than baseline models across multiple sessions. |
Empathy in Diversity: Personalized Depression and Anxiety Therapy via Dialogue State Tracking and Patient-Aware Planning (2026.acl-long)
Copied to clipboard
Xinwei Yang, Junyi Fan, Yuqing Liu, Jiaxuan Wang, Jiashuai Zhang, Hongru Liang, Wenqiang Lei, Yao Song
| Challenge: | Recent efforts have turned to large language models (LLMs) as therapeutic agents for psychological therapy tasks, yet robustness across diverse patients remains underexplored. |
| Approach: | They propose a realistic role-play protocol for evaluating therapeutic dialogue agents and a de-identified, expert-annotated corpus of therapist–patient dialogues. |
| Outcome: | The proposed framework outperforms baselines on therapeutic outcomes and dialogue quality while improving conversational efficiency. |
Measuring What Matters!! Assessing Therapeutic Principles in Mental-Health Conversation (2026.acl-long)
Copied to clipboard
| Challenge: | Recent systems exhibit conversational competence but lack structured mechanisms to evaluate adherence to core therapeutic principles. |
| Approach: | They propose a framework to evaluate therapist-like responses for clinically grounded appropriateness and effectiveness using an ordinal scale. |
| Outcome: | The proposed framework achieves an F-1 score of 63.34 versus the baseline Qwen3 score of 38.56 . |
CBT-Bench: Evaluating Large Language Models on Assisting Cognitive Behavior Therapy (2025.naacl-long)
Copied to clipboard
Mian Zhang, Xianjun Yang, Xinlu Zhang, Travis Labrum, Jamie C. Chiu, Shaun M. Eack, Fei Fang, William Yang Wang, Zhiyu Chen
| Challenge: | Existing research has explored mental health condition classifications, empathetic conversations, and chatbots designed for simple discourse structures. |
| Approach: | They propose a benchmark for systematic evaluation of cognitive behavioral therapy assistance using Large Language Models (LLMs). |
| Outcome: | The proposed benchmark includes three levels of tasks covering key aspects of cognitive behavioral therapy that could be enhanced through AI assistance. |
Do Large Language Models Align with Core Mental Health Counseling Competencies? (2025.findings-naacl)
Copied to clipboard
Viet Cuong Nguyen, Mohammad Taher, Dongwan Hong, Vinicius Konkolics Possobom, Vibha Thirunellayi Gopalakrishnan, Ekta Raj, Zihang Li, Heather J. Soled, Michael L. Birnbaum, Srijan Kumar, Munmun De Choudhury
| Challenge: | Large language models are promising for mental health, but their alignment with core counseling competencies remains underexplored. |
| Approach: | They propose a benchmark to evaluate 22 general-purpose and medical-finetuned LLMs across five key competencies. |
| Outcome: | The proposed model outperforms generalist models in Intake, Assessment & Diagnosis but struggles with core counseling attributes and professional practice & ethics. |
EmoRes: Toward Adaptive Psychological Support via User-Agnostic Benchmark and Topic-Mining Agent (2026.findings-acl)
Copied to clipboard
Zhengwei Zou, Xuanming Jiang, Baoyi An, Dingyu Nie, Zhengxing Fang, Qingyu Liu, Xueming Qian, Guoshuai Zhao, Zhongyu Yang
| Challenge: | Large language models generate fragmented and emotionally inconsistent dialogues lacking the therapeutic structure necessary for reliable assessment. |
| Approach: | They propose a framework that boosts psychological reasoning via a Topic-Mining Emotional Agent and a multi-perspective Self-Reflection Agent. |
| Outcome: | The proposed framework improves topic continuity, emotional coherence, and clinical interpretability over baselines and validated by ablation studies and human evaluations. |
Cactus: Towards Psychological Counseling Conversations using Cognitive Behavioral Theory (2024.findings-emnlp)
Copied to clipboard
Suyeon Lee, Sunghwan Kim, Minju Kim, Dongjin Kang, Dongil Yang, Harim Kim, Minseok Kang, Dayi Jung, Min Kim, Seungbeen Lee, Kyong-Mee Chung, Youngjae Yu, Dongha Lee, Jinyoung Yeo
| Challenge: | Existing models that use large language models are not available due to ethical concerns, and data privacy concerns are a concern. |
| Approach: | They propose a multi-turn dialogue dataset that emulates real-life counseling interactions using the goal-oriented approach of Cognitive Behavioral Therapy (CBT). |
| Outcome: | The proposed model outperforms other models in counseling skills, highlighting its effectiveness and potential as a counseling agent. |
Eeyore: Realistic Depression Simulation via Expert-in-the-Loop Supervised and Preference Optimization (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have been explored for mental healthcare training and therapy client simulation, but they fail to authentically capture diverse client traits and psychological conditions. |
| Approach: | They propose an 8B model optimized for realistic depression simulation with expert input at every stage. |
| Outcome: | The model outperforms GPT-4o in linguistic authenticity and profile adherence. |
When Can We Trust LLMs in Mental Health? Large-Scale Benchmarks for Reliable LLM Evaluation (2026.eacl-long)
Copied to clipboard
Abeer Badawi, Elahe Rahimi, Md Tahmid Rahman Laskar, Sheri Grach, Lindsay Bertrand, Lames Danok, Prathiba Dhanesh, Jimmy Huang, Frank Rudzicz, Elham Dolatabadi
| Challenge: | Existing benchmarks for large language models are limited in scale, authenticity, and reliability due to the emotionally complex nature of therapeutic dialogue. |
| Approach: | They propose two benchmarks that provide a framework for evaluating large language models for mental health support. |
| Outcome: | The proposed framework provides a framework for generation and evaluation of large-scale authentic dialogue datasets and judge-reliability assessments. |
CRADLE Bench: A Clinician-Annotated Benchmark for Multi-Faceted Mental Health Crisis and Safety Risk Detection (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing language models fail to detect high-risk situations such as suicide ideation and child abuse . |
| Approach: | They propose a benchmark for multi-faceted mental health crisis detection that incorporates temporal labels. |
| Outcome: | The proposed benchmark significantly outperforms single-model annotations on social media posts and development examples. |