Papers by Shaoting Zhang
MedOdyssey: A Medical Domain Benchmark for Long Context Evaluation Up to 200K Tokens (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing benchmarks in the generic domain have evaluated long-context capabilities for LLMs. |
| Approach: | They propose a medical long-context benchmark with seven length levels ranging from 4K to 200K tokens. |
| Outcome: | The proposed benchmarks have seven length levels ranging from 4K to 200K tokens. |
MMXU: A Multi-Modal and Multi-X-ray Understanding Dataset for Disease Progression (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing datasets and models fail to consider critical aspects of medical diagnostics, authors argue . MMXU enables multi-image questions incorporating both current and historical patient data. |
| Approach: | They propose a dataset for MedVQA that focuses on identifying changes in specific regions between two patient visits. |
| Outcome: | The proposed dataset improves diagnostic accuracy by 20% by integrating historical data. |
Alleviating Exposure Bias via Multi-level Contrastive Learning and Deviation Simulation in Abstractive Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Abstractive summarization systems have a severe mismatch between training and inference, i.e., exposure bias. |
| Approach: | They propose a multi-level contrastive learning framework for abstractive summarization and a tailored sparse decoder self-attention pattern to bridge the gap between training and inference. |
| Outcome: | The proposed framework outperforms the state-of-the-art models on two summarization datasets while adding relatively low overhead. |
Guiding Medical Vision-Language Models with Diverse Visual Prompts: Framework Design and Comprehensive Exploration of Prompt Variations (2025.naacl-long)
Copied to clipboard
| Challenge: | Current vision-language models lack the ability to focus on specific areas designated by humans . a new framework that integrates medical entity extraction, visual prompt generation, and dataset adaptation is proposed to improve visual prompt-guided fine-tuning. |
| Approach: | They propose to use visual prompts to guide and enhance formation of region-specific attention. |
| Outcome: | The proposed framework outperforms state-of-the-art large vision-language models on medical datasets. |
GraphDx: A Cost-Aware Knowledge-Enhanced Multi-Agent Framework for Sequential Diagnosis (2026.findings-acl)
Copied to clipboard
Shaoting Tan, Ning Liu, Yuntao Du, Shuyue Wei, Wu Shuai, Qian Li, Yanyu Xu, Wei Zhang, Lizhen Cui, Haitao Yuan
| Challenge: | Existing Large Language Models struggle to reason systematically under cost constraints . Existing approaches lack the knowledge-reasoning capability to reason under cost . |
| Approach: | They propose a knowledge-enhanced framework that leverages large language models to construct MDKGs . they propose three collaborative agents that handle language understanding and generation . |
| Outcome: | GraphDx improves diagnostic success rates from 50–68% to 79–93% while reducing test costs by 20–54%. |
DiagnosisArena: Benchmarking Diagnostic Reasoning for Large Language Models (2026.findings-acl)
Copied to clipboard
Yakun Zhu, Zhongzhen Huang, Linjie Mu, Yutong Huang, Wei Nie, Jiaji Liu, Shaoting Zhang, Pengfei Liu, Xiaofan Zhang
| Challenge: | Existing medical benchmarks for diagnostic reasoning are limited in their ability to perform complex tasks. |
| Approach: | They propose to benchmark diagnostic capabilities of large language models to assess their accuracy and generalization bottlenecks. |
| Outcome: | The proposed model achieves 45.82%, 31.09%, and 17.79% accuracy, compared to current models, o3-mini, e1 and DeepSeek-R1 . |
Meta-Tool: Unleash Open-World Function Calling Capabilities of General-Purpose Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models struggle with addressing diverse user inquiries in open-world tasks. |
| Approach: | They propose a plug-and-play tool retrieval system for LLMs to access external tool library and use retrieved tools to solve user's problem. |
| Outcome: | The proposed model improves on a finetuned version of LLaMA-3.1 and 2,800 dialogues and 7,361 tools spanning ten distinct test categories. |
Medical Dialogue System: A Survey of Categories, Methods, Evaluation and Challenges (2024.findings-acl)
Copied to clipboard
Xiaoming Shi, Zeming Liu, Li Du, Yuxuan Wang, Hongru Wang, Yuhang Guo, Tong Ruan, Jie Xu, Xiaofan Zhang, Shaoting Zhang
| Challenge: | Existing medical dialogue systems have significant potential to simplify diagnostic procedure and reduce the cost of collecting information from patients. |
| Approach: | They analyze 325 papers from well-known computer science, natural language processing conferences and journals to find out the major challenges of medical dialog systems. |
| Outcome: | The proposed systems have been surveyed in the medical community but have not been evaluated from a technical perspective. |
MidMed: Towards Mixed-Type Dialogues for Medical Consultation (2023.acl-long)
Copied to clipboard
| Challenge: | Current medical dialogue systems assume that patients have explicit goals but are often unavailable in real-world situations due to the lack of medical knowledge. |
| Approach: | They propose a human-to-human mixed-type medical consultation dialogue corpus . they build benchmarking baselines on MidMed and propose an instruction-guiding framework . Experimental results show the effectiveness of InsMed . |
| Outcome: | The proposed system can help patients clarify their goals in real-world situations . it covers four departments with 8,309 dialogues and provides benchmarking baselines . |
MeNTi: Bridging Medical Calculator and LLM Agent with Nested Tool Calling (2025.naacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have been widely used in medicine but are limited in their ability to fully address the complexities of the real world. |
| Approach: | They propose a universal agent architecture for Large Language Models that integrates a specialized medical toolkit and employs meta-tool and nested calling mechanisms to enhance LLM tool utilization. |
| Outcome: | The proposed framework improves the accuracy and performance of medical calculators in complex medical scenarios. |
GECSum: Generative Evaluation-Driven Sequence Level Contrastive Learning for Abstractive Summarization (2024.lrec-main)
Copied to clipboard
| Challenge: | Abstractive summarization is a technique in natural language processing that involves generating a summary of a source document by creating new sentences and phrases. |
| Approach: | They propose a sequence-level contrastive learning framework that leverages the semantic understanding capabilities of the abstractive model itself to evaluate summary in reference-based settings. |
| Outcome: | The proposed framework outperforms the state-of-the-art in four summarization datasets. |
Interactive Evaluation for Medical LLMs via Task-oriented Dialogue System (2025.coling-main)
Copied to clipboard
| Challenge: | In typical medical scenarios, doctors often ask a set of questions to gain a comprehensive understanding of patients’ conditions. |
| Approach: | They propose to use multi-turn medical dialogue evaluation to evaluate proactive communication and diagnostic capabilities of medical Large Language Models (LLMs) . |
| Outcome: | The proposed model outperforms existing models on multi-turn question-answering datasets and is therefore cost-effective. |
MedMCP-Calc: Benchmarking LLMs for Realistic Medical Calculator Scenarios via MCP Integration (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks focus on static single-step calculations with explicit instructions. |
| Approach: | They propose a benchmark for evaluating medical calculators in realistic scenarios . they use 118 scenario tasks across 4 clinical domains to evaluate medical calculator performance . |
| Outcome: | The first benchmark for evaluating medical calculators in realistic scenarios is released . it features 118 scenario tasks across 4 clinical domains and is based on a model context protocol integration. |