Format Inertia: A Failure Mechanism of LLMs in Medical Pre-Consultation (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models have brought significant improvements to various service domains, including chatbots and medical pre-consultation applications. |
| Approach: | They propose a method that rebalances the turn-count distribution of training data to mitigate Format Inertia in medical pre-consultation tasks. |
| Outcome: | The proposed method significantly alleviates Format Inertia in medical pre-consultation tasks. |
Similar Papers
Reasoning Is Not All You Need: Examining LLMs for Multi-Turn Mental Health Conversations (2026.acl-long)
Copied to clipboard
| Challenge: | Existing evaluation frameworks focus on diagnostic accuracy and win-rates and often overlook alignment with patient-specific goals, values, and personalities required for meaningful conversations. |
| Approach: | They propose a framework for synthetically generating realistic, multi-turn mental health sensemaking conversations and a dataset to examine their models in healthcare settings. |
| Outcome: | The proposed framework synthesizes a dataset comprising over 2,200 patient–LLM conversations and evaluates them using human-centric criteria. |
Assessing and Mitigating Medical Knowledge Drift and Conflicts in Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Rapid medical concept drift can lead LLMs to provide incorrect or outdated advice. |
| Approach: | They propose to evaluate how large language models manage knowledge conflicts in clinical guidelines. |
| Outcome: | The proposed benchmark evaluates how LLMs manage varied knowledge conflicts in clinical guidelines. |
Do LLMs Understand Dialogues? A Case Study on Dialogue Acts (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown remarkable performance on many unseen tasks in a zero-shot setting. |
| Approach: | They propose to identify three key pre-tasks essential for accurate DA prediction: Turn Management, Communicative Function Identification, and Dialogue Structure Prediction. |
| Outcome: | The proposed model fails to outperform basic rule-based tasks on three key pre-tasks, and the results suggest that the model is flawed. |
MedRedFlag: Investigating how LLMs Redirect Misconceptions in Real-World Health Communication (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly being used by lay users for medical advice, but they have not yet been tested for this crucial competency. |
| Approach: | They develop a semi-automated pipeline to curate MedRedFlag, a dataset of 1100+ reddit questions that require redirection. |
| Outcome: | The proposed pipeline compares state-of-the-art LLMs to those from clinicians to find out how they perform under real-world health communication. |
A Survey of LLM-based Agents in Medicine: How far are we from Baymax? (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are transforming healthcare through their ability to understand and assist with medical tasks. |
| Approach: | They analyze system profiles, clinical planning, medical reasoning frameworks, and external capacity enhancement. |
| Outcome: | The findings highlight the future directions in medical reasoning, physical system integration, and training simulations. |
Can LLMs Replace Clinical Doctors? Exploring Bias in Disease Diagnosis by Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | a new study examines the bias of disease prediction in large language models . the model biases are prevalent across gender, age range and disease judgment behaviors . |
| Approach: | They propose a prompt-based approach to alleviate the bias in disease prediction with LLMs. |
| Outcome: | The proposed model alleviates the observed bias in disease prediction with LLMs. |
Facts Fade Fast: Evaluating Memorization of Outdated Medical Knowledge in Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | LLMs encode extensive knowledge within their parameters, but the knowledge in LLM models can become outdated over time. |
| Approach: | They propose two new LLMs that provide outdated medical advice . they compare the models with a set of QA pairs whose verdict changed through time . |
| Outcome: | The proposed models exhibit memorization of outdated knowledge to some extent. |
Inertia in Moral and Value Judgments of Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models behave non-deterministically, and prompting is a common method for steering their outputs. |
| Approach: | They use role-play at scale to study the value orientation and inertia of Large Language Models. |
| Outcome: | The proposed model keeps values skewed in one direction across persona settings. |
Knowing When to Abstain: Medical LLMs Under Clinical Uncertainty (2026.eacl-long)
Copied to clipboard
Sravanthi Machcha, Sushrita Yerra, Sahil Gupta, Aishwarya Sahoo, Sharmin Sultana, Hong Yu, Zonghai Yao
| Challenge: | Current evaluation of large language models prioritizes accuracy, but abstention is vital for trustworthy deployment. |
| Approach: | They propose a benchmark and evaluation protocol for abstention in medical multiple-choice question answering . they integrate conformal prediction, adversarial question perturbations, and explicit abstraction options. |
| Outcome: | The proposed protocol improves reliability of medical multiple-choice question answering models by providing explicit abstention options. |
DrAgent: Empowering Large Language Models as Medical Agents for Multi-hop Medical Reasoning (2025.findings-emnlp)
Copied to clipboard
Fenglin Liu, Zheng Li, Hongjian Zhou, Qingyu Yin, Jingfeng Yang, Xin Liu, Zhengyang Wang, Xianfeng Tang, Shiyang Li, Xiang He, Ruijie Wang, Bing Yin, Xiao Gu, Lei Clifton, David A. Clifton
| Challenge: | commercial LLMs can be difficult to use in real-world clinical decision-making . a lightweight LLM can be used to collaborate with diverse clinical tools . |
| Approach: | They propose a lightweight LLM that can be used to build medical LLMs as agents . they use recursive curriculum learning to optimize the LLM in an easy-to-hard progression . |
| Outcome: | The proposed approach outperforms human experts in medical examinations on diverse datasets. |