Papers with PubMedQA
MediSwift: Efficient Sparse Pre-trained Biomedical Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models are typically trained on general source data forvarious domains, but domain-specific pre-training is expensive and requires computational costs. |
| Approach: | They propose a suite of biomedicalLMs that leverage sparse pre-training on domain-specific biomedically text data. |
| Outcome: | The proposed model outperforms existing LLMs on biomedical tasks by 22.5x . |
MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning (2024.findings-acl)
Copied to clipboard
Xiangru Tang, Anni Zou, Zhuosheng Zhang, Ziming Li, Yilun Zhao, Xingyao Zhang, Arman Cohan, Mark Gerstein
| Challenge: | Large language models face unique challenges such as domain-specific terminologies and reasoning over specialized knowledge. |
| Approach: | They propose a multi-disciplinary collaboration framework that leverages LLM-based agents in a role-playing setting. |
| Outcome: | The proposed framework excels at mining and harnessing medical expertise within LLMs, as well as extending its reasoning abilities. |
A Continued Pretrained LLM Approach for Automatic Medical Note Generation (2024.naacl-short)
Copied to clipboard
Dong Yuan, Eti Rastogi, Gautam Naik, Sree Prasanna Rajagopal, Sagar Goyal, Fen Zhao, Bharath Chintagunta, Jeffrey Ward
| Challenge: | HEAL is the first continuously trained LLaMA2-based LLM for medical conversations . despite the success of LLMs in general capabilities, they often fall short in niche domains like healthcare . |
| Approach: | They propose a 13B LLaMA2-based LLM that is purpose-built for medical conversations and measured on automated scribing. |
| Outcome: | The HEAL LLM outperforms GPT-4 and PMC-LLaMA in PubMedQA with 78.4% accuracy and parity with GPT-LLAMA in generating medical notes. |
Reasoning or Knowledge: Stratified Evaluation of Biomedical LLMs (2026.eacl-long)
Copied to clipboard
| Challenge: | Medical reasoning in large language models is a complex cognitive process through which clinicians interpret patient data and make diagnostic and therapeutic decisions. |
| Approach: | They propose an evaluation framework that disentangles knowledge recall from reasoning by training a PubMedBERT-based classifier and applying it to 11 widely used biomedical QA benchmarks. |
| Outcome: | The proposed evaluation framework disentangles knowledge recall from reasoning by training a PubMedBERT-based classifier and applying it to 11 widely used biomedical QA benchmarks. |
Hierarchical Representation-based Dynamic Reasoning Network for Biomedical Question Answering (2022.coling-1)
Copied to clipboard
Jianguo Mao, Jiyuan Zhang, Zengfeng Zeng, Weihua Peng, Wenbin Jiang, Xiangdong Wang, Hong Liu, Yajuan Lyu
| Challenge: | Existing models of biomedical question answering are limited in their ability to predict answers . a new model improves the performance of existing models, but the code will be released after the paper is published. |
| Approach: | They propose a hierarchical representation-based dynamic reasoning network to solve biomedical problems. |
| Outcome: | The proposed model significantly improves on three mainstream biomedical datasets . the code will be released after the paper is published . |
MedHallu: A Comprehensive Benchmark for Detecting Medical Hallucinations in Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) generate plausible but factually incorrect outputs, posing serious risks to patient safety and clinical decision-making. |
| Approach: | They propose a benchmark for medical hallucination detection using 10,000 question-answer pairs derived from PubMedQA. |
| Outcome: | The proposed model achieves an F1 score as low as 0.625 for detecting 'hard' category hallucinations. |
Dialogue is Better Than Monologue: Instructing Meidcal LLMs via Strategic Conversations (2026.findings-eacl)
Copied to clipboard
Zijie Liu, Xinyu Zhao, Jie Peng, Jinhao Duan, Zhuangdi Zhu, Qingyu Chen, Kaidi Xu, Xia Hu, Tianlong Chen
| Challenge: | Existing tuning methods for medical AI models are monologue-based . existing benchmarks are based on licensing exams or research articles . |
| Approach: | They propose a benchmark to expose limitations of monologue-based tuning for medical AI models . they use a large dialogue dataset to capture stepwise diagnostic reasoning . |
| Outcome: | The proposed model outperforms monologue-tuned models on a medical question answering task and improves accuracy on standard medical QA benchmarks. |
PubMedQA: A Dataset for Biomedical Research Question Answering (D19-1)
Copied to clipboard
| Challenge: | PubMedQA is a biomedical question answering dataset based on PubMed abstracts . 68.1% accuracy is achieved, compared to single human performance of 78.0% . |
| Approach: | They propose a biomedical question answering dataset from PubMed abstracts . the dataset is annotated by experts and has 1k instances of QA . |
| Outcome: | The proposed model achieves 68.1% accuracy compared to human performance of 78.0% and majority-baseline of 55.2%. |
Multi-hop Inference for Question-driven Summarization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for summarizing source document for non-factoid questions are lacking in factoidic QA. |
| Approach: | They propose a question-driven abstractive summarization method that incorporates multi-hop reasoning into question-based summarizing. |
| Outcome: | The proposed method outperforms state-of-the-art methods on two non-factoid QA datasets. |
PubMed Reasoner: Dynamic Reasoning-based Retrieval for Evidence-Grounded Biomedical Question Answering (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to QA provide inaccurate answers but lack mechanisms to iteratively refine poor queries. |
| Approach: | They propose a biomedical question answering agent that performs self-critic query refinement . they propose re-reflection methods that kick in only after full retrieval is completed . |
| Outcome: | a biomedical question answering agent achieves 78.32% accuracy on PubMedQA . the proposed approach provides practical assistance to clinicians and biomedically researchers . |
VE-KD: Vocabulary-Expansion Knowledge-Distillation for Training Smaller Domain-Specific Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | VE-KD is a method that balances knowledge distillation and vocabulary expansion with the aim of training efficient domain-specific language models. |
| Approach: | They propose a method that balances knowledge distillation and vocabulary expansion with the aim of training efficient domain-specific language models. |
| Outcome: | VE-KD outperforms DistilBERT and Adapt-and-Distill in biomedical domain tasks . compared with other methods, it outperformed Distilbert and adapted-and distill . |
CMedCalc-Bench: A Fine-Grained Benchmark for Chinese Medical Calculations in LLM (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing medical NLP benchmarks focus on qualitative reasoning and textual comprehension, but lack of fine-grained evaluation of intermediate reasoning. |
| Approach: | They propose a Chinese medical calculation benchmark that disentangles clinical entity extraction from numerical computation. |
| Outcome: | The proposed framework disentangles clinical entity extraction from numerical computation, enabling systematic diagnosis of model deficiencies. |
ReasonMed: A 370K Multi-Agent Generated Dataset for Advancing Medical Reasoning (2025.emnlp-main)
Copied to clipboard
Yu Sun, Xingyu Qian, Weiwen Xu, Hao Zhang, Chenghao Xiao, Long Li, Deli Zhao, Wenbing Huang, Tingyang Xu, Qifeng Bai, Yu Rong
| Challenge: | Existing medical reasoning datasets are limited in scale and typically rely on incomplete data. |
| Approach: | They propose to use ReasonMed to train medical reasoning models using a multi-agent generation, verification, and refinement pipeline. |
| Outcome: | The largest medical reasoning dataset to date surpasses the prior best sub-10B models by 4.17% and even exceeds LLaMA3.1-70B on PubMedQA by 4.60%. |
MDTeamGPT: Mitigating Context Collapse and Enabling Self-Evolution in Medical Multi-Agent Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Long, multi-round, multirole interaction trajectories lead to severe information dilution and context window overload, triggering context collapse which destabilizes reasoning. |
| Approach: | They propose a multi-agent framework that compresses and reorganizes multi-round consensus. |
| Outcome: | The proposed framework outperforms baselines across text-based and multimodal tasks while demonstrating superior diagnostic performance and stability in complex clinical scenarios. |