Papers by Xiaofan Zhang
MedOdyssey: A Medical Domain Benchmark for Long Context Evaluation Up to 200K Tokens (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing benchmarks in the generic domain have evaluated long-context capabilities for LLMs. |
| Approach: | They propose a medical long-context benchmark with seven length levels ranging from 4K to 200K tokens. |
| Outcome: | The proposed benchmarks have seven length levels ranging from 4K to 200K tokens. |
MMXU: A Multi-Modal and Multi-X-ray Understanding Dataset for Disease Progression (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing datasets and models fail to consider critical aspects of medical diagnostics, authors argue . MMXU enables multi-image questions incorporating both current and historical patient data. |
| Approach: | They propose a dataset for MedVQA that focuses on identifying changes in specific regions between two patient visits. |
| Outcome: | The proposed dataset improves diagnostic accuracy by 20% by integrating historical data. |
RESF: Regularized-Entropy-Sensitive Fingerprinting for Black-Box Tamper Detection of Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for tamper detection rely on model stability, not inherently stochastic models. |
| Approach: | They propose a hypothesis-testing method for black-box tamper detection for LLMs . they propose regularized entropy-sensitive fingerprinting to enable efficient fingerprinting . |
| Outcome: | The proposed method achieves 98.80% detection accuracy under challenging conditions . it is based on a first-order surrogate for KL divergence to identify prompts most responsive to parameter perturbations. |
Alleviating Exposure Bias via Multi-level Contrastive Learning and Deviation Simulation in Abstractive Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Abstractive summarization systems have a severe mismatch between training and inference, i.e., exposure bias. |
| Approach: | They propose a multi-level contrastive learning framework for abstractive summarization and a tailored sparse decoder self-attention pattern to bridge the gap between training and inference. |
| Outcome: | The proposed framework outperforms the state-of-the-art models on two summarization datasets while adding relatively low overhead. |
SyntheT2C: Generating Synthetic Data for Fine-Tuning Large Language Models on the Text2Cypher Task (2025.coling-main)
Copied to clipboard
| Challenge: | Existing efforts to bolster LLMs’ proficiency in Cypher generation are hindered by the lack of annotated datasets of Query-Cypher pairs. |
| Approach: | They propose a method for constructing a synthetic Query-Cypher pair dataset using LLM prompting and template-filling. |
| Outcome: | The proposed method enhances the performance of LLMs on Text2Cypher task via SFT. |
An LLM-based Framework for Biomedical Terminology Normalization in Social Media via Multi-Agent Collaboration (2025.coling-main)
Copied to clipboard
| Challenge: | Experimental results indicate that our approach exhibits competitive performance. |
| Approach: | They propose a tuning-free approach to normalize non-standard terms using large language models . they use a search engine and a domain knowledge base to expand the short texts into accurate descriptions . |
| Outcome: | The proposed approach is based on the "Recall and Re-rank" framework . it can be used to identify the standard term in a specified termbase for non-standardized mentions . |
DiagnosisArena: Benchmarking Diagnostic Reasoning for Large Language Models (2026.findings-acl)
Copied to clipboard
Yakun Zhu, Zhongzhen Huang, Linjie Mu, Yutong Huang, Wei Nie, Jiaji Liu, Shaoting Zhang, Pengfei Liu, Xiaofan Zhang
| Challenge: | Existing medical benchmarks for diagnostic reasoning are limited in their ability to perform complex tasks. |
| Approach: | They propose to benchmark diagnostic capabilities of large language models to assess their accuracy and generalization bottlenecks. |
| Outcome: | The proposed model achieves 45.82%, 31.09%, and 17.79% accuracy, compared to current models, o3-mini, e1 and DeepSeek-R1 . |
Meta-Tool: Unleash Open-World Function Calling Capabilities of General-Purpose Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models struggle with addressing diverse user inquiries in open-world tasks. |
| Approach: | They propose a plug-and-play tool retrieval system for LLMs to access external tool library and use retrieved tools to solve user's problem. |
| Outcome: | The proposed model improves on a finetuned version of LLaMA-3.1 and 2,800 dialogues and 7,361 tools spanning ten distinct test categories. |
Medical Dialogue System: A Survey of Categories, Methods, Evaluation and Challenges (2024.findings-acl)
Copied to clipboard
Xiaoming Shi, Zeming Liu, Li Du, Yuxuan Wang, Hongru Wang, Yuhang Guo, Tong Ruan, Jie Xu, Xiaofan Zhang, Shaoting Zhang
| Challenge: | Existing medical dialogue systems have significant potential to simplify diagnostic procedure and reduce the cost of collecting information from patients. |
| Approach: | They analyze 325 papers from well-known computer science, natural language processing conferences and journals to find out the major challenges of medical dialog systems. |
| Outcome: | The proposed systems have been surveyed in the medical community but have not been evaluated from a technical perspective. |
MidMed: Towards Mixed-Type Dialogues for Medical Consultation (2023.acl-long)
Copied to clipboard
| Challenge: | Current medical dialogue systems assume that patients have explicit goals but are often unavailable in real-world situations due to the lack of medical knowledge. |
| Approach: | They propose a human-to-human mixed-type medical consultation dialogue corpus . they build benchmarking baselines on MidMed and propose an instruction-guiding framework . Experimental results show the effectiveness of InsMed . |
| Outcome: | The proposed system can help patients clarify their goals in real-world situations . it covers four departments with 8,309 dialogues and provides benchmarking baselines . |
ESF: Efficient Sensitive Fingerprinting for Black-Box Tamper Detection of Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly utilized in diverse applications, including code generation, legal document analysis, medical diagnosis, and decision-making. |
| Approach: | They propose a fingerprinting method tailored for black-box tamper detection of large language models. |
| Outcome: | The proposed method detects tampering with a 99.2% detection rate using 5 fingerprint samples across state-of-the-art LLMs. |
Mix-of-Granularity: Optimize the Chunking Granularity for Retrieval-Augmented Generation (2025.coling-main)
Copied to clipboard
| Challenge: | Retrieval-augmented generation systems often use a fixed strategy to extract information from multiple sources. |
| Approach: | They propose a method that dynamically determines optimal granularity of a knowledge source based on input queries using a router. |
| Outcome: | The proposed method predicts optimal granularity levels and significantly improves performance in downstream tasks. |
MeNTi: Bridging Medical Calculator and LLM Agent with Nested Tool Calling (2025.naacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have been widely used in medicine but are limited in their ability to fully address the complexities of the real world. |
| Approach: | They propose a universal agent architecture for Large Language Models that integrates a specialized medical toolkit and employs meta-tool and nested calling mechanisms to enhance LLM tool utilization. |
| Outcome: | The proposed framework improves the accuracy and performance of medical calculators in complex medical scenarios. |
GECSum: Generative Evaluation-Driven Sequence Level Contrastive Learning for Abstractive Summarization (2024.lrec-main)
Copied to clipboard
| Challenge: | Abstractive summarization is a technique in natural language processing that involves generating a summary of a source document by creating new sentences and phrases. |
| Approach: | They propose a sequence-level contrastive learning framework that leverages the semantic understanding capabilities of the abstractive model itself to evaluate summary in reference-based settings. |
| Outcome: | The proposed framework outperforms the state-of-the-art in four summarization datasets. |
Interactive Evaluation for Medical LLMs via Task-oriented Dialogue System (2025.coling-main)
Copied to clipboard
| Challenge: | In typical medical scenarios, doctors often ask a set of questions to gain a comprehensive understanding of patients’ conditions. |
| Approach: | They propose to use multi-turn medical dialogue evaluation to evaluate proactive communication and diagnostic capabilities of medical Large Language Models (LLMs) . |
| Outcome: | The proposed model outperforms existing models on multi-turn question-answering datasets and is therefore cost-effective. |
MedMCP-Calc: Benchmarking LLMs for Realistic Medical Calculator Scenarios via MCP Integration (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks focus on static single-step calculations with explicit instructions. |
| Approach: | They propose a benchmark for evaluating medical calculators in realistic scenarios . they use 118 scenario tasks across 4 clinical domains to evaluate medical calculator performance . |
| Outcome: | The first benchmark for evaluating medical calculators in realistic scenarios is released . it features 118 scenario tasks across 4 clinical domains and is based on a model context protocol integration. |
Tracing Training Footprints: A Calibration Approach for Membership Inference Attacks Against Multimodal Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to improve difficulty calibration for Multimodal Large Language Models only consider text input . visual embeddings in training data reduce effectiveness of these methods . |
| Approach: | They propose a method to detect member samples in poorly generalized local manifolds by visual embeddings. |
| Outcome: | The proposed method surpasses existing methods. |