Unveiling Performance Challenges of Large Language Models in Low-Resource Healthcare: A Demographic Fairness Perspective (2025.coling-main)
Copied to clipboard
| Challenge: | Existing large language models (LLMs) are not effective in solving real-world healthcare tasks, but they are able to provide demographic information and provide biased health predictions. |
| Approach: | They evaluate state-of-the-art LLMs with three prevalent learning frameworks across six diverse healthcare tasks and find significant challenges in applying LLM to real-world healthcare tasks. |
| Outcome: | The proposed models perform poorly in real-world healthcare tasks and are inconsistent with existing learning frameworks. |
Similar Papers
Missing the Margins: A Systematic Literature Review on the Demographic Representativeness of LLMs (2025.findings-acl)
Copied to clipboard
| Challenge: | 211 studies on the demographic representativeness of large language models have conflicting results . 29% of the studies report positive conclusions on the representativeness, 30% do not evaluate LLMs across multiple demographic categories or within demographic subcategories. |
| Approach: | 211 papers review the representativeness of large language models . authors recommend more precise evaluation methods and comprehensive documentation of demographic attributes . |
| Outcome: | 211 studies on the representativeness of large language models are reviewed . 29% of the studies report positive conclusions, but 30% fail to specify subcategories . authors recommend more precise evaluation methods and documentation of demographic attributes . |
The Impossibility of Fair LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Existing frameworks for evaluating large language models do not extend to general-purpose AI contexts or are infeasible in practice. |
| Approach: | They analyze a variety of technical fairness frameworks to find inherent challenges . they find that each framework does not logically extend to the general-purpose AI context . |
| Outcome: | The proposed frameworks do not logically extend to the general-purpose AI context or are infeasible in practice due to large amounts of unstructured training data and potential combinations of human populations, use cases, and sensitive attributes. |
A Comprehensive Survey on the Trustworthiness of Large Language Models in Healthcare (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a survey of large language models in healthcare raises critical concerns around trustworthiness . trustworthy of LLMs in healthcare remains underexplored, lacking a systematic review . |
| Approach: | a new survey examines the trustworthiness of large language models in healthcare . a review examines how each dimension affects reliability and ethical deployment of LLMs . |
| Outcome: | The present study examines the trustworthiness of large language models in healthcare . it identifies key gaps in existing approaches and challenges posed by evolving paradigms . |
Can LLMs Replace Clinical Doctors? Exploring Bias in Disease Diagnosis by Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | a new study examines the bias of disease prediction in large language models . the model biases are prevalent across gender, age range and disease judgment behaviors . |
| Approach: | They propose a prompt-based approach to alleviate the bias in disease prediction with LLMs. |
| Outcome: | The proposed model alleviates the observed bias in disease prediction with LLMs. |
How Can We Diagnose and Treat Bias in Large Language Models for Clinical Decision-Making? (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent studies have shown that LLMs exhibit social biases inherited from training data. |
| Approach: | They propose a framework for evaluation and mitigation of bias in Large Language Models applied to complex clinical cases using a dataset based on the JAMA Clinical Challenge. |
| Outcome: | The proposed framework employs multiple choice questions and explanations to evaluate gender and ethnicity biases in LLMs. |
Everything you need to know about Multilingual LLMs: Towards fair, performant and reliable models for languages of the world (2023.acl-tutorials)
Copied to clipboard
| Challenge: | Responsible AI issues such as fairness, bias and toxicity will be discussed in this tutorial . |
| Approach: | This tutorial will describe various aspects of scaling up language technologies to many of the world’s languages by describing the latest research in Massively Multilingual Language Models (MMLMs). |
| Outcome: | This tutorial will cover various aspects of scaling up language technologies to many of the world's languages by describing the latest research in multilingual models. |
A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendations (2024.emnlp-main)
Copied to clipboard
Md Tahmid Rahman Laskar, Sawsan Alqahtani, M Saiful Bari, Mizanur Rahman, Mohammad Abdullah Matin Khan, Haidar Khan, Israt Jahan, Amran Bhuiyan, Chee Wei Tan, Md Rizwan Parvez, Enamul Hoque, Shafiq Joty, Jimmy Huang
| Challenge: | Large Language Models (LLMs) have gained significant attention due to their capabilities in performing diverse tasks across domains. |
| Approach: | They review the primary challenges and limitations causing inconsistencies in evaluations . early models could generate coherent text but limited to simple tasks . |
| Outcome: | The proposed evaluations are reproducible, reliable, and robust. |
Large Language Models Are Poor Clinical Decision-Makers: A Comprehensive Benchmark (2024.emnlp-main)
Copied to clipboard
Fenglin Liu, Zheng Li, Hongjian Zhou, Qingyu Yin, Jingfeng Yang, Xianfeng Tang, Chen Luo, Ming Zeng, Haoming Jiang, Yifan Gao, Priyanka Nigam, Sreyashi Nag, Bing Yin, Yining Hua, Xuan Zhou, Omid Rohanian, Anshul Thakur, Lei Clifton, David Clifton
| Challenge: | Existing studies focus on evaluating large language models in close-ended QA tasks, but many clinical decisions involve answering open-ended questions without pre-set options. |
| Approach: | They construct a benchmark to better understand large language models in the clinic . they use existing datasets to evaluate LLMs in clinical situations . |
| Outcome: | The proposed model outperforms human experts in multiple medical tasks. |
Large Language Models Are Still Misled by Simple Bias Ensembles (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks for large language models are constrained to datasets where each sample is manually injected with only one type of bias. |
| Approach: | They propose a multi-bias benchmark where each sample contains multiple types of biases. |
| Outcome: | The proposed benchmark shows that existing LLMs and debiasing methods perform poorly on this benchmark, highlighting the challenge of eliminating compounded biases. |
Assessing and Mitigating Medical Knowledge Drift and Conflicts in Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Rapid medical concept drift can lead LLMs to provide incorrect or outdated advice. |
| Approach: | They propose to evaluate how large language models manage knowledge conflicts in clinical guidelines. |
| Outcome: | The proposed benchmark evaluates how LLMs manage varied knowledge conflicts in clinical guidelines. |