Papers with MIMIC-CXR
The Impact of Auxiliary Patient Data on Automated Chest X-Ray Report Generation and How to Incorporate It (2025.acl-long)
Copied to clipboard
| Challenge: | Traditionally, CXR report generation relies on data from a patient’s exam, overlooking valuable information from patient electronic health records. |
| Approach: | They propose to integrate patient data from ED records into multimodal language models that embed patient data into a language model. |
| Outcome: | The proposed model incorporates patient data from the MIMIC-CXR and MIMICIV-ED datasets to improve diagnostic accuracy and improves radiologist effectiveness. |
Language over Labels: Contrastive Language Supervision Exceeds Purely Label-Supervised Classification Performance on Chest X-Rays (2022.aacl-srw)
Copied to clipboard
| Challenge: | Pretrained CLIP models lack domain-specific knowledge of text and images. |
| Approach: | They adapt CLIP-based models to the chest radiography domain using contrastive language supervision and a detailed ablation study of the batch and dataset size. |
| Outcome: | The proposed model outperforms supervised learning on labels on the MIMIC-CXR dataset while generalizing to the CheXpert and RSNA Pneumonia datasets. |
Coherent and Concise Radiology Report Generation via Context Specific Image Representations and Orthogonal Sentence States (2021.naacl-industry)
Copied to clipboard
| Challenge: | Neural models for text generation are often designed in an end-to-end fashion, limiting their practical usability in downstream applications. |
| Approach: | They propose a method to compute image representations specific to each sentential context and exploiting diverse sentence states to ensure topical continuity and content diversity of generated radiology reports. |
| Outcome: | The proposed method outperforms baselines on objective metrics and human evaluations by 18% and 29% respectively in the evaluation for informativeness and content ordering respectively. |
Reinforced Cross-modal Alignment for Radiology Report Generation (2022.findings-acl)
Copied to clipboard
| Challenge: | Medical images are widely used in clinical decision-making, where writing radiology reports can be enhanced by automatic solutions to alleviate physicians’ workload. |
| Approach: | They propose an approach with reinforcement learning over a cross-modal memory to better align visual and textual features for radiology report generation. |
| Outcome: | The proposed approach improves cross-modal alignment on two English radiology report datasets and human evaluation confirms the results. |
Multimodal Generation of Radiology Reports using Knowledge-Grounded Extraction of Entities and Relations (2022.aacl-main)
Copied to clipboard
Francesco Dalla Serra, William Clackett, Hamish MacKinnon, Chaoyang Wang, Fani Deligianni, Jeff Dalton, Alison Q. O’Neil
| Challenge: | Existing approaches to generate text radiology reports are prone to errors and poor clinical accuracy. |
| Approach: | They propose a two-step pipeline that subdivides the problem into factual triple extraction followed by free-text report generation. |
| Outcome: | The proposed pipeline shows that the generated reports exhibit realistic style but lack clinical accuracy. |
Semantic Consistency-Based Uncertainty Quantification for Factuality in Radiology Report Generation (2025.findings-naacl)
Copied to clipboard
| Challenge: | Radiology report generation has shown great potential in assisting radiologists . generative medical Vision Large Language Models (VLLMs) are prone to hallucinations and can produce inaccurate diagnostic information. |
| Approach: | They propose a framework that provides both report-level and sentence-level uncertainties. |
| Outcome: | The proposed method improves factuality scores by 10% by rejecting 20% of reports on the MIMIC-CXR dataset. |
Generating Radiology Reports via Memory-driven Transformer (2020.emnlp-main)
Copied to clipboard
| Challenge: | Medical imaging reports are time-consuming and can be error-prone for inexperienced radiologists. |
| Approach: | They propose to generate radiology reports with memory-driven Transformer using relational memory and memory-based conditional layer normalization. |
| Outcome: | The proposed method outperforms existing models on IU X-Ray and MIMIC-CXR . it generates long reports with medical terms and meaningful image-text attention mappings . |
GPT-4V Cannot Generate Radiology Reports Yet (2025.findings-naacl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are becoming multimodal, and GPT-4 models are supposed to possess advanced skills across a wide range of domains, including high-stakes scenarios such as medicine. |
| Approach: | They perform a systematic evaluation of GPT-4 in generating radiology reports across three chest X-ray report benchmarks: MIMIC-CXR, CheXpert Plus, and IU X ray. |
| Outcome: | The proposed model fails in lexical and clinical efficacy metrics . the distributions of model-predicted labels remain constant regardless of groundtruth conditions on the image, suggesting that the model is not interpreting chest X-rays meaningfully. |
Normal-Abnormal Decoupling Memory for Medical Report Generation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for capturing nuanced visual information are prone to data bias and noise. |
| Approach: | They propose a normal-abnormal semantic decoupling network that utilizes abnormal pattern memory to optimize visual extraction through the extraction of abnormal semantics from the reports. |
| Outcome: | The proposed approach surpasses the current state-of-the-art methods on the benchmark MIMIC-CXR and shows excellent performance on the same dataset. |
Attend to Medical Ontologies: Content Selection for Clinical Abstractive Summarization (2020.acl-main)
Copied to clipboard
| Challenge: | Existing studies have reported that clinicians read the IMPRESSION as they have less time to review findings. |
| Approach: | They propose to augment salient ontological terms into the abstractive summarizer by augmenting salient ontologies into the semantic summariser. |
| Outcome: | The proposed model significantly improves state-of-the-art results in terms of ROUGE metrics on two publicly available clinical data sets. |
Cross-modal Contrastive Attention Model for Medical Report Generation (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods for medical report generation are unable to capture useful information from historical cases. |
| Approach: | They propose a model that captures both visual and semantic information from similar cases. |
| Outcome: | The proposed model outperforms the state-of-the-art models on almost all metrics on IU X-Ray and MIMIC-CXR benchmarks. |
DDGIP: Radiology Report Generation Through Disease Description Graph and Informed Prompting (2025.findings-naacl)
Copied to clipboard
| Challenge: | Automatic radiology report generation is challenging due to inherent biases in medical imaging data. |
| Approach: | They propose a disease description graph that encapsulates comprehensive and pertinent disease information. |
| Outcome: | The proposed model outperforms state-of-the-art models on two widely-used datasets . the proposed model is based on a three-layer decoder and improves on existing models . |
Competence-based Multimodal Curriculum Learning for Medical Report Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Medical report generation is more challenging for data-driven neural models due to data bias and limited medical data. |
| Approach: | They propose a Competence-based Multimodal Curriculum Learning framework to alleviate the data bias by efficiently utilizing the limited medical data for medical report generation. |
| Outcome: | The proposed framework can be incorporated into existing models to improve their performance on the IU-Xray and MIMIC-CXR datasets. |
RA-RRG: Multimodal Retrieval-Augmented Radiology Report Generation with Key Phrase Extraction (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing MLLMs are computationally expensive and may produce hallucinated content . RA-RRG uses large language models to generate radiology reports . |
| Approach: | They propose a retrieval-augmented RRG framework that combines multimodal retrieval with large language models to generate radiology reports. |
| Outcome: | RA-RRG uses large language models to generate radiology reports . it suppresses hallucinations while maintaining strong report generation performance . |
DeltaNet: Conditional Medical Report Generation for COVID-19 Diagnosis (2022.coling-1)
Copied to clipboard
Xian Wu, Shuxin Yang, Zhaopeng Qiu, Shen Ge, Yangtian Yan, Xingwang Wu, Yefeng Zheng, S. Kevin Zhou, Li Xiao
| Challenge: | X-ray and CT are the gold standard for COVID-19 diagnosis and treatment . however, due to the excessive number of patients, writing reports becomes a heavy burden for radiologists. |
| Approach: | They propose to use X-ray and CT to generate medical reports automatically . they evaluate DeltaNet on a COVID-19 dataset, where it outperforms state-of-the-art approaches . |
| Outcome: | The proposed system outperforms state-of-the-art methods on a COVID-19 dataset. |
Graph Enhanced Contrastive Learning for Radiology Findings Summarization (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for automating impression generation have limited the relationship between extra knowledge and the original findings. |
| Approach: | They propose a framework for automating impression generation that exploits extra knowledge and original findings . they propose combining key words and their relations to extract critical information . |
| Outcome: | The proposed framework exploits extra knowledge and the original findings in an integrated way . the state-of-the-art results on two datasets confirm the effectiveness of the proposed method . |
Controllable Chest X-Ray Report Generation from Longitudinal Representations (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Radiology reports are detailed text descriptions of the content of medical scans. |
| Approach: | They propose a method to align, concatenate and fuse the current and prior visual information into a joint longitudinal representation which can be provided to a multimodal report generation model. |
| Outcome: | The proposed method achieves state-of-the-art results while enabling anatomy-wise controllable report generation. |
Writing by Memorizing: Hierarchical Retrieval-based Medical Report Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for medical image analysis use predefined template databases or ignore hierarchical nature of medical report generation. |
| Approach: | They propose a hierarchical retrieval mechanism to extract both report and sentence-level templates for clinically accurate report generation. |
| Outcome: | The proposed model extracts both report and sentence-level templates for clinically accurate report generation. |
Structuring Radiology Reports: Challenging LLMs with Lightweight Models (2025.emnlp-main)
Copied to clipboard
Johannes Moll, Louisa Fay, Asfandyar Azhar, Sophie Ostmeier, Sergios Gatidis, Tim C. Lueth, Curtis Langlotz, Jean-Benoit Delbrouck
| Challenge: | Radiology reports lack a standardized format, limiting both interpretability and machine learning applications. |
| Approach: | They propose to use lightweight encoder-decoder models for structuring radiology reports . they compare models with eight open-source LLMs with prompting and in-context learning . |
| Outcome: | The proposed models outperform eight open-source LLMs on a human-annotated test set. |
Word Graph Guided Summarization for Radiology Findings (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing studies focus on introducing salient word information to general text summarization framework to guide selection of key content in radiology findings. |
| Approach: | They propose a method for automatic impression generation using word graphs and a Word Graph guided Summarization model to capture critical words and their relations. |
| Outcome: | The proposed method is validated on two datasets, OPENI and MIMIC-CXR. |
Cross-modal Memory Networks for Radiology Report Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Medical imaging reports are essential in clinical practice, and generating the reports is beneficial to reduce the burden of radiologists. |
| Approach: | They propose to use a shared memory to enhance the encoder-decoder framework for radiology report generation. |
| Outcome: | The proposed model can generate more accurate reports on two widely used datasets. |
Dynamic Knowledge Prompt for Chest X-ray Report Generation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for radiology report generation fail to incorporate prior knowledge . data bias, sparse features of chest X-ray image make it difficult to generate reports . |
| Approach: | They propose a dynamically integrated framework for chest X-ray report generation that incorporates pulmonary lesion knowledge at the instance-level. |
| Outcome: | The proposed framework can dynamically incorporate pulmonary lesion knowledge at instance-level to facilitate report generation. |
CHiLL: Zero-shot Custom Interpretable Feature Extraction from Clinical Notes with Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study suggests that linear models with interpretable features are more reliable than opaque models. |
| Approach: | They propose an approach for natural-language specification of features for linear models . they prompt LLMs with expert-crafted queries to generate interpretable features from health records . |
| Outcome: | The proposed approach can be used to craft features clinically meaningful for downstream tasks . it is based on a risk prediction task and standard predictive tasks based upon this data . |
CmEAA: Cross-modal Enhancement and Alignment Adapter for Radiology Report Generation (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods for automatic radiology report generation suffer from data bias. |
| Approach: | They propose a method that connects a vision encoder with a frozen large language model by using a cross-modal enhancement and alignment adapter. |
| Outcome: | The proposed model outperforms existing state-of-the-art methods on IU X-Ray and MIMIC-CXR datasets. |
Replace and Report: NLP Assisted Radiology Report Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Clinical practice frequently uses medical imaging for diagnosis and treatment. |
| Approach: | They propose a template-based approach to generate radiology reports from radiographs . they use multilabel image classifiers to generate tags, pathological descriptions from tags . |
| Outcome: | The proposed method improves on the most popular radiology report datasets. |
CLEAR: A Clinically Grounded Tabular Framework for Radiology Report Evaluation (2025.findings-emnlp)
Copied to clipboard
Yuyang Jiang, Chacha Chen, Shengyuan Wang, Feng Li, Zecong Tang, Benjamin M. Mervak, Lydia Chelala, Christopher M Straus, Reve Chahine, Samuel G. Armato Iii, Chenhao Tan
| Challenge: | Existing metrics lack the granularity and interpretability to capture nuanced clinical differences between candidate and ground-truth radiology reports. |
| Approach: | They propose a tabular framework with E**xpert-curated labels and an attribute-level comparison for radiology report evaluation (**CLEAR) |
| Outcome: | The proposed framework can extract clinical attributes and provide automated metrics that are strongly aligned with clinical judgment. |
Libra: Leveraging Temporal Images for Biomedical Radiology Analysis (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for radiology report generation rely on single-image analysis or rule-based heuristics to process multiple images. |
| Approach: | They propose a temporal-aware MLLM tailored for chest X-ray report generation that combines a radiology-specific image encoder with a novel Temporal Alignment Connector. |
| Outcome: | The proposed model sets new standards in clinical relevance and lexical accuracy on the MIMIC-CXR dataset. |
Fine-grained Medical Vision-Language Representation Learning for Radiology Report Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to learn medical vision-language representations by contrasting images with entire reports are not effective. |
| Approach: | They propose a phenotype-driven medical vision-language representation learning framework to bridge the gap between visual and textual modalities for improved text-oriented generation. |
| Outcome: | The proposed framework bridges the gap between visual and textual modalities for improved radiology report generation. |
A Self-training Framework for Automated Medical Report Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Medical report generation is an important medical artificial intelligence task. |
| Approach: | They propose a framework for medical report generation that exploits unlabeled medical images and a reference-free evaluation metric. |
| Outcome: | The proposed framework performs better than previous fully-supervised models trained on entire training data. |
RADAR: Enhancing Radiology Report Generation with Supplementary Knowledge Injection (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to enhance radiology report generation overlook the knowledge already embedded within the models, leading to redundant information integration. |
| Approach: | They propose a framework for enhancing radiology report generation with supplementary knowledge injection that leverages both internal and external knowledge. |
| Outcome: | Extensive experiments on MIMIC-CXR, CheXpert-Plus, and IU X-ray show that the proposed model outperforms state-of-the-art LLMs in both language quality and clinical accuracy. |
Automated Structured Radiology Report Generation (2025.acl-long)
Copied to clipboard
Jean-Benoit Delbrouck, Justin Xu, Johannes Moll, Alois Thomas, Zhihong Chen, Sophie Ostmeier, Asfandyar Azhar, Kelvin Zhenghao Li, Andrew Johnston, Christian Bluethgen, Eduardo Pontes Reis, Mohamed S Muneer, Maya Varma, Curtis Langlotz
| Challenge: | Existing models struggle to produce consistent, clinically meaningful reports and standard evaluation metrics fail to capture the nuances of radiological interpretation. |
| Approach: | They propose to reformulate free-text radiology reports into a standardized format, ensuring clarity, consistency, and structured clinical reporting. |
| Outcome: | The proposed task reformulates free-text radiology reports into a standardized format, ensuring clarity, consistency, and structured clinical reporting. |
CheXalign: Preference fine-tuning in chest X-ray interpretation models without human feedback (2025.acl-long)
Copied to clipboard
Dennis Hein, Zhihong Chen, Sophie Ostmeier, Justin Xu, Maya Varma, Eduardo Pontes Reis, Arne Edward Michalson Md, Christian Bluethgen, Hyun Joo Shin, Curtis Langlotz, Akshay S Chaudhari
| Challenge: | Radiologists are a crucial role in translating medical images into actionable reports . however, the field faces staffing shortages and increasing workloads . |
| Approach: | They propose an automated pipeline for preference feedback focusing on chest X-ray radiology report generation (RRG) method leverages publicly available datasets containing pairs of images and radiologist-written reference reports with reference-based metrics, or Judges. |
| Outcome: | The proposed pipeline achieves state-of-the-art CheXbert scores on the MIMIC-CXR dataset while on average maintaining robust performance across six additional image perception and reasoning tasks. |
CSTRL: Context-Driven Sequential Transfer Learning for Abstractive Radiology Report Summarization (2025.findings-acl)
Copied to clipboard
Mst. Fahmida Sultana Naznin, Adnan Ibney Faruq, Mostafa Rifat Tazwar, Md Jobayer, Md. Mehedi Hasan Shawon, Md Rakibul Hasan
| Challenge: | Pretrained models that excel in abstractive summarization problems face challenges when applied to specialized medical domains due to complex terminology and the necessity for accurate clinical context. |
| Approach: | They propose a sequential transfer learning model that ensures key content extraction and coherent summarization. |
| Outcome: | The proposed model shows 56.2% improvement in BLEU-1, 40.5% in ble-2, 84.3% in blu-3, 28.9% in ROUGE-1, 41.0% in Rough-2 and 26.5% of ROGUE-3 over benchmark studies. |