CBT-Bench: Evaluating Large Language Models on Assisting Cognitive Behavior Therapy (2025.naacl-long)
Copied to clipboard
Mian Zhang, Xianjun Yang, Xinlu Zhang, Travis Labrum, Jamie C. Chiu, Shaun M. Eack, Fei Fang, William Yang Wang, Zhiyu Chen
| Challenge: | Existing research has explored mental health condition classifications, empathetic conversations, and chatbots designed for simple discourse structures. |
| Approach: | They propose a benchmark for systematic evaluation of cognitive behavioral therapy assistance using Large Language Models (LLMs). |
| Outcome: | The proposed benchmark includes three levels of tasks covering key aspects of cognitive behavioral therapy that could be enhanced through AI assistance. |
Similar Papers
CBT-LLM: A Chinese Large Language Model for Cognitive Behavioral Therapy-based Mental Health Question Answering (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent advances in artificial intelligence highlight the potential of language models in psychological health support. |
| Approach: | They propose a method to enhance the precision and efficacy of psychological support through large language models. |
| Outcome: | The proposed model generates professional and structured responses in Chinese psychological health Q&A tasks, showcasing its practicality and quality. |
Can AI Relate: Testing Large Language Model Response for Mental Health Support (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) are already being piloted for clinical use in hospitals . recent failures of the Tessa chatbot have led to doubts about their reliability in high-stakes settings. |
| Approach: | They propose safety guidelines for the potential deployment of large language models for mental health response. |
| Outcome: | The proposed framework measures equity in empathy and adherence of LLM responses to motivational interviewing theory. |
Large Language Models Are Poor Clinical Decision-Makers: A Comprehensive Benchmark (2024.emnlp-main)
Copied to clipboard
Fenglin Liu, Zheng Li, Hongjian Zhou, Qingyu Yin, Jingfeng Yang, Xianfeng Tang, Chen Luo, Ming Zeng, Haoming Jiang, Yifan Gao, Priyanka Nigam, Sreyashi Nag, Bing Yin, Yining Hua, Xuan Zhou, Omid Rohanian, Anshul Thakur, Lei Clifton, David Clifton
| Challenge: | Existing studies focus on evaluating large language models in close-ended QA tasks, but many clinical decisions involve answering open-ended questions without pre-set options. |
| Approach: | They construct a benchmark to better understand large language models in the clinic . they use existing datasets to evaluate LLMs in clinical situations . |
| Outcome: | The proposed model outperforms human experts in multiple medical tasks. |
A Survey of Large Language Models in Psychotherapy: Current Landscape and Future Directions (2025.findings-acl)
Copied to clipboard
Hongbin Na, Yining Hua, Zimu Wang, Tao Shen, Beibei Yu, Lilin Wang, Wei Wang, John Torous, Ling Chen
| Challenge: | Large language models (LLMs) can handle extensive context and multi-turn reasoning. |
| Approach: | They propose a taxonomy dividing psychotherapy into stages of assessment, diagnosis, and treatment to examine LLM advancements and challenges. |
| Outcome: | The proposed taxonomy reveals imbalances in current research, such as a focus on common disorders, linguistic biases, fragmented methods, and limited theoretical integration. |
When Can We Trust LLMs in Mental Health? Large-Scale Benchmarks for Reliable LLM Evaluation (2026.eacl-long)
Copied to clipboard
Abeer Badawi, Elahe Rahimi, Md Tahmid Rahman Laskar, Sheri Grach, Lindsay Bertrand, Lames Danok, Prathiba Dhanesh, Jimmy Huang, Frank Rudzicz, Elham Dolatabadi
| Challenge: | Existing benchmarks for large language models are limited in scale, authenticity, and reliability due to the emotionally complex nature of therapeutic dialogue. |
| Approach: | They propose two benchmarks that provide a framework for evaluating large language models for mental health support. |
| Outcome: | The proposed framework provides a framework for generation and evaluation of large-scale authentic dialogue datasets and judge-reliability assessments. |
Using Linguistic Entrainment to Evaluate Large Language Models for Use in Cognitive Behavioral Therapy (2025.findings-naacl)
Copied to clipboard
Mina Kian, Kaleen Shrestha, Katrin Fischer, Xiaoyuan Zhu, Jonathan Ong, Aryan Trehan, Jessica Wang, Gloria Chang, Séb Arnold, Maja Mataric
| Challenge: | Entrainment is a communication process that builds a strong relationship between a mental health therapist and their client. |
| Approach: | They evaluate the linguistic entrainment of an LLM in a mental health dialog setting and compare it to trained therapists and non-expert online peer supporters. |
| Outcome: | The proposed model outperforms humans in a cognitive behavioral therapy setting. |
From Conversation to Automation: Leveraging LLMs for Problem-Solving Therapy Analysis (2025.findings-acl)
Copied to clipboard
| Challenge: | Problem-Solving Therapy (PST) is a structured psychological approach that helps individuals manage stress and resolve personal issues. |
| Approach: | They developed a framework for PST annotation using established PST Core Strategies and a set of novel Facilitative Strategies to analyze a corpus of real-world therapy transcripts to determine which strategies are most prevalent. |
| Outcome: | The proposed framework outperforms existing models and LLMs to identify the most prevalent strategies in a corpus of real-world therapy transcripts. |
PATIENT-𝜓: Using Large Language Models to Simulate Patients for Training Mental Health Professionals (2024.emnlp-main)
Copied to clipboard
Ruiyi Wang, Stephanie Milani, Jamie Chiu, Jiayin Zhi, Shaun Eack, Travis Labrum, Samuel Murphy, Nev Jones, Kate Hardy, Hong Shen, Fei Fang, Zhiyu Chen
| Challenge: | Mental illness remains one of the most critical public health issues. |
| Approach: | They propose a patient simulation framework for cognitive behavior therapy training that uses large language models to act as a simulated therapy patient. |
| Outcome: | The proposed framework improves the skill acquisition and confidence of mental health trainees beyond textbooks, videos, and role-play with non-patients. |
Do Large Language Models Align with Core Mental Health Counseling Competencies? (2025.findings-naacl)
Copied to clipboard
Viet Cuong Nguyen, Mohammad Taher, Dongwan Hong, Vinicius Konkolics Possobom, Vibha Thirunellayi Gopalakrishnan, Ekta Raj, Zihang Li, Heather J. Soled, Michael L. Birnbaum, Srijan Kumar, Munmun De Choudhury
| Challenge: | Large language models are promising for mental health, but their alignment with core counseling competencies remains underexplored. |
| Approach: | They propose a benchmark to evaluate 22 general-purpose and medical-finetuned LLMs across five key competencies. |
| Outcome: | The proposed model outperforms generalist models in Intake, Assessment & Diagnosis but struggles with core counseling attributes and professional practice & ethics. |
HealMe: Harnessing Cognitive Reframing in Large Language Models for Psychotherapy (2024.acl-long)
Copied to clipboard
Mengxi Xiao, Qianqian Xie, Ziyan Kuang, Zhicheng Liu, Kailai Yang, Min Peng, Weiguang Han, Jimin Huang
| Challenge: | Large Language Models (LLMs) can be used in psychotherapy to overcome challenges such as shame, distrust, and resource scarcity. |
| Approach: | They propose a cognitive reframing therapy method that uses empathetic dialogue to address deep-rooted negative thoughts and fosters rational, balanced perspectives. |
| Outcome: | The proposed model outperforms other models in terms of empathy, guidance, and logical coherence, demonstrating its effectiveness and potential positive impact on psychotherapy. |