Mohammad Anas Azeez, Rafiq Ali, Ebad Shabbir, Zohaib Hasan Siddiqui, Gautam Siddharth Kashyap, Jiechao Gao, Usman Naseem
| Challenge: | Large Language Models (LLMs) are promising for transforming digital health applications . but ensuring they meet industry standards for factual accuracy, usefulness, and safety remains a challenge . |
| Approach: | They present a framework to assess large language models' accuracy, usefulness, and safety . they assess models' honesty, helpfulness, harmlessness and domain-specific tuning . |
| Outcome: | The proposed framework assesses models across honesty, helpfulness, and harmlessness . AlpaCare-13B achieves highest accuracy (91.7%) and harmlessity (0.92) . |
Similar Papers
Are LLMs reliable? An exploration of the reliability of large language models in clinical note generation (2025.acl-industry)
Copied to clipboard
Kristine Ann M. Carandang, Jasper Meynard Arana, Ethan Robert Casin, Christopher Monterola, Daniel Stanley Tan, Jesus Felix B. Valenzuela, Christian Alis
| Challenge: | Clinical note generation (CNG) tools are being developed to address extended working hours and healthcare provider fatigue. |
| Approach: | They evaluate the reliability of 12 open-weight and proprietary LLMs from Anthropic, Meta, Mistral, and OpenAI in CNG in terms of their ability to generate notes that are string equivalent (consistency rate), have the same meaning (semantic consistency) and are correct (symbol similarity) |
| Outcome: | The results show that the LLMs generated notes that are string equivalent (consistency rate), have the same meaning (semantic consistency) and are correct (symbol similarity) overall, Meta’s Llama 70B was the most reliable, followed by Mistral’s Small model. |
Health-ORSC-Bench: A Benchmark for Measuring Over-Refusal and Safety Completion in Health Context (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing safety alignment benchmarks fail to evaluate Safe Completion: the model’s ability to maximise helpfulness on dual-use or borderline queries without crossing into actionable harm. |
| Approach: | They propose a large-scale benchmark to measure Over-Refusal and Safe Completion quality in healthcare. |
| Outcome: | The framework evaluates 30 state-of-the-art LLMs including GPT-5 and Claude-4. |
A Comprehensive Survey on the Trustworthiness of Large Language Models in Healthcare (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a survey of large language models in healthcare raises critical concerns around trustworthiness . trustworthy of LLMs in healthcare remains underexplored, lacking a systematic review . |
| Approach: | a new survey examines the trustworthiness of large language models in healthcare . a review examines how each dimension affects reliability and ethical deployment of LLMs . |
| Outcome: | The present study examines the trustworthiness of large language models in healthcare . it identifies key gaps in existing approaches and challenges posed by evolving paradigms . |
Trustworthy Medical Question Answering: An Evaluation-Centric Survey (2025.emnlp-main)
Copied to clipboard
Yinuo Wang, Baiyang Wang, Robert Mercer, Frank Rudzicz, Sudipta Singha Roy, Pengjie Ren, Zhumin Chen, Xindi Wang
| Challenge: | achieving comprehensive trustworthiness in medical QA poses significant challenges due to complexity of healthcare data, critical nature of clinical scenarios, and multifaceted dimensions of trustworthy AI. |
| Approach: | They examine six key dimensions of trustworthiness in medical QA . they compare how each dimension is evaluated in existing LLM-based systems . |
| Outcome: | The findings show that large language models have improved patient safety and effectiveness . the models exhibit critical trust failures when deployed in clinical settings . |
From Scores to Steps: Diagnosing and Improving LLM Performance in Evidence-Based Medical Calculations (2025.emnlp-main)
Copied to clipboard
Benlu Wang, Iris Xia, Yifan Zhang, Junda Wang, null Feiyun Ouyang, Shuo Han, Arman Cohan, Hong Yu, Zonghai Yao
| Challenge: | Existing benchmarks assess only the final answer with a wide numerical tolerance, overlooking systematic reasoning failures and potentially causing serious clinical misjudgments. |
| Approach: | They propose a new step-by-step evaluation pipeline that assesses formula selection, entity extraction, and arithmetic computation. |
| Outcome: | The proposed method improves the accuracy of large language models on medical benchmarks from 16.35% to 53.19%. |
BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domains (2024.findings-acl)
Copied to clipboard
Yanis Labrak, Adrien Bazoge, Emmanuel Morin, Pierre-Antoine Gourraud, Mickael Rouvier, Richard Dufour
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable versatility in recent years, offering potential applications across specialized domains such as healthcare and medicine. |
| Approach: | They propose an open-source LLM tailored for the biomedical domain that utilizes Mistral as its foundation model and pre-trained on PubMed Central. |
| Outcome: | The proposed model outperforms existing models on a benchmark comprising 10 established medical question-answering tasks in English and is competitive with proprietary models. |
Automatic Evaluation of Healthcare LLMs Beyond Question-Answering (2025.naacl-short)
Copied to clipboard
Anna Arias-Duart, Pablo Agustin Martin-Torres, Daniel Hinjos, Pablo Bernabeu-Perez, Lucia Urcelay Ganzabal, Marta Gonzalez Mallo, Ashwin Kumar Gururajan, Enrique Lopez-Cuena, Sergio Alvarez-Napagao, Dario Garcia-Gasulla
| Challenge: | Current Large Language Models (LLMs) benchmarks are often based on open-ended or close-ended QA evaluations, avoiding the requirement of human labor. |
| Approach: | They propose a multi-axis suite for healthcare LLM evaluation, exploring correlations between open and close benchmarks and metrics. |
| Outcome: | The proposed framework explores correlations between open and close benchmarks and metrics in the healthcare domain, with blind spots and overlaps in existing methodologies. |
Beyond the Leaderboard: Rethinking Medical Benchmarks for Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are proving significant potential in healthcare, prompting numerous benchmarks to evaluate their capabilities. |
| Approach: | They propose a framework that deconstructs benchmark development into five stages from design to governance and provides a checklist of 46 medically-tailored criteria. |
| Outcome: | The framework deconstructs benchmark development into five stages from design to governance and provides a comprehensive checklist of 46 medically-tailored criteria. |
When Can We Trust LLMs in Mental Health? Large-Scale Benchmarks for Reliable LLM Evaluation (2026.eacl-long)
Copied to clipboard
Abeer Badawi, Elahe Rahimi, Md Tahmid Rahman Laskar, Sheri Grach, Lindsay Bertrand, Lames Danok, Prathiba Dhanesh, Jimmy Huang, Frank Rudzicz, Elham Dolatabadi
| Challenge: | Existing benchmarks for large language models are limited in scale, authenticity, and reliability due to the emotionally complex nature of therapeutic dialogue. |
| Approach: | They propose two benchmarks that provide a framework for evaluating large language models for mental health support. |
| Outcome: | The proposed framework provides a framework for generation and evaluation of large-scale authentic dialogue datasets and judge-reliability assessments. |
Benchmarking LLMs on Authentic Cases from Medical Journals (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing medical benchmarks suffer from performance saturation due to medical exam questions. |
| Approach: | They evaluate the performance of over 20 open-source and proprietary large language models and benchmark them against human medical experts. |
| Outcome: | The new benchmark is based on authentic clinical cases sourced from medical journals and implements rigorous human review process to ensure the quality and reliability of the benchmark. |