Controllable Chest X-Ray Report Generation from Longitudinal Representations

Challenge:	Radiology reports are detailed text descriptions of the content of medical scans.
Approach:	They propose a method to align, concatenate and fuse the current and prior visual information into a joint longitudinal representation which can be provided to a multimodal report generation model.
Outcome:	The proposed method achieves state-of-the-art results while enabling anatomy-wise controllable report generation.

Similar Papers

Overview

Challenge

Approach

Outcome

Multimodal Generation of Radiology Reports using Knowledge-Grounded Extraction of Entities and Relations (2022.aacl-main)

Copied to clipboard

Francesco Dalla Serra, William Clackett, Hamish MacKinnon, Chaoyang Wang, Fani Deligianni, Jeff Dalton, Alison Q. O’Neil

Information Extraction & Text Mining

Challenge:	Existing approaches to generate text radiology reports are prone to errors and poor clinical accuracy.
Approach:	They propose a two-step pipeline that subdivides the problem into factual triple extraction followed by free-text report generation.
Outcome:	The proposed pipeline shows that the generated reports exhibit realistic style but lack clinical accuracy.

The Impact of Auxiliary Patient Data on Automated Chest X-Ray Report Generation and How to Incorporate It (2025.acl-long)

Copied to clipboard

Aaron Nicolson, Shengyao Zhuang, Jason Dowling, Bevan Koopman

Semantic Text Processing

Challenge:	Traditionally, CXR report generation relies on data from a patient’s exam, overlooking valuable information from patient electronic health records.
Approach:	They propose to integrate patient data from ED records into multimodal language models that embed patient data into a language model.
Outcome:	The proposed model incorporates patient data from the MIMIC-CXR and MIMICIV-ED datasets to improve diagnostic accuracy and improves radiologist effectiveness.

Fine-grained Medical Vision-Language Representation Learning for Radiology Report Generation (2023.emnlp-main)

Copied to clipboard

Siyuan Wang, Bo Peng, Yichao Liu, Qi Peng

Visual Data in NLP

Challenge:	Existing methods to learn medical vision-language representations by contrasting images with entire reports are not effective.
Approach:	They propose a phenotype-driven medical vision-language representation learning framework to bridge the gap between visual and textual modalities for improved text-oriented generation.
Outcome:	The proposed framework bridges the gap between visual and textual modalities for improved radiology report generation.

Automated Structured Radiology Report Generation (2025.acl-long)

Copied to clipboard

Jean-Benoit Delbrouck, Justin Xu, Johannes Moll, Alois Thomas, Zhihong Chen, Sophie Ostmeier, Asfandyar Azhar, Kelvin Zhenghao Li, Andrew Johnston, Christian Bluethgen, Eduardo Pontes Reis, Mohamed S Muneer, Maya Varma, Curtis Langlotz

Language Models

Challenge:	Existing models struggle to produce consistent, clinically meaningful reports and standard evaluation metrics fail to capture the nuances of radiological interpretation.
Approach:	They propose to reformulate free-text radiology reports into a standardized format, ensuring clarity, consistency, and structured clinical reporting.
Outcome:	The proposed task reformulates free-text radiology reports into a standardized format, ensuring clarity, consistency, and structured clinical reporting.

Replace and Report: NLP Assisted Radiology Report Generation (2023.findings-acl)

Copied to clipboard

Kaveri Kale, Pushpak Bhattacharyya, Kshitij Jadhav

Information Retrieval

Challenge:	Clinical practice frequently uses medical imaging for diagnosis and treatment.
Approach:	They propose a template-based approach to generate radiology reports from radiographs . they use multilabel image classifiers to generate tags, pathological descriptions from tags .
Outcome:	The proposed method improves on the most popular radiology report datasets.

Coherent and Concise Radiology Report Generation via Context Specific Image Representations and Orthogonal Sentence States (2021.naacl-industry)

Copied to clipboard

Litton J Kurisinkel, Ai Ti Aw, Nancy F Chen

Multimodality

Challenge:	Neural models for text generation are often designed in an end-to-end fashion, limiting their practical usability in downstream applications.
Approach:	They propose a method to compute image representations specific to each sentential context and exploiting diverse sentence states to ensure topical continuity and content diversity of generated radiology reports.
Outcome:	The proposed method outperforms baselines on objective metrics and human evaluations by 18% and 29% respectively in the evaluation for informativeness and content ordering respectively.

Automated Generation of Accurate & Fluent Medical X-ray Reports (2021.emnlp-main)

Copied to clipboard

Hoang Nguyen, Dong Nie, Taivanbat Badamdorj, Yujie Liu, Yingying Zhu, Jason Truong, Li Cheng

Multimodality

Challenge:	Existing medical report generation efforts focus on producing human-readable reports, yet the generated text may not be well aligned to the clinical facts.
Approach:	They propose to automate the generation of medical reports from chest X-ray image inputs . medical reports are the primary medium, which physicians communicate findings from scans - authors say .
Outcome:	The proposed method achieves fluency and clinical accuracy on common metrics.

Enhancing Image-to-Text Generation in Radiology Reports through Cross-modal Multi-Task Learning (2024.lrec-main)

Copied to clipboard

Nurbanu Aksoy, Nishant Ravikumar, Serge Sharoff

Multimodality

Challenge:	Image-to-text generation relies on independent models for image understanding and natural language generation, which often exhibit a semantic gap between visual and textual information.
Approach:	They propose a multi-task learning framework to leverage both visual and non-imaging data for generating radiology reports.
Outcome:	The proposed framework improves performance over single-task baselines across language generation metrics and mitigates overfitting in auxiliary tasks.

Learning Visual-Semantic Embeddings for Reporting Abnormal Findings on Chest X-rays (2020.findings-emnlp)

Copied to clipboard

Jianmo Ni, Chun-Nan Hsu, Amilcare Gentili, Julian McAuley

Multimodality

Challenge:	Existing work on report generation often trains encoder-decoder networks to generate complete reports, but such models are affected by data bias and face common issues inherent in text generation models.
Approach:	They propose a method to identify abnormal findings from radiology images and group them with unsupervised clustering and minimal rules.
Outcome:	The proposed method outperforms existing generation models on correctness and text generation metrics.

Divide and Conquer Radiology Report Generation via Observation Level Fine-grained Pretraining and Prompt Tuning (2024.emnlp-main)

Copied to clipboard

Yuanpin Zhou, Huogen Wang

Language Models

Challenge:	Recent advances in image captioning and vision-language pretraining have made it difficult for radiologists to generate coherent and accurate reports.
Approach:	They propose a model which breaks down full-text radiology reports into concise observation descriptions and encodes observation predictions into a decoding stage.
Outcome:	The proposed model achieves significant improvements across all metrics, underscoring its capability to generate semantically coherent and clinically accurate radiology reports.