Look & Mark: Leveraging Radiologist Eye Fixations and Bounding boxes in Multimodal Large Language Models for Chest X-ray Report Generation (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in multimodal Large Language Models (LLMs) have significantly enhanced the automation of medical image analysis, but still suffer from hallucinations and clinically significant errors. |
| Approach: | They propose a grounding fixation strategy that integrates radiologist eye fixations and bounding box annotations into the LLM prompting framework. |
| Outcome: | The proposed model improves performance without retraining across domain-specific and general-purpose models and achieves an 87.3% clinical average performance. |
Similar Papers
CCD: Mitigating Hallucinations in Radiology MLLMs via Clinical Contrastive Decoding (2026.findings-acl)
Copied to clipboard
| Challenge: | Multimodal large language models generate medical hallucinations due to over-sensitivity to clinical sections. |
| Approach: | They propose a framework that integrates structured clinical signals from task-specific radiology expert models. |
| Outcome: | The proposed framework improves overall performance on radiology report generation (RRG) on the MIMIC-CXR dataset, it yields up to 17% improvement in RadGraph-F1. |
Structuring Radiology Reports: Challenging LLMs with Lightweight Models (2025.emnlp-main)
Copied to clipboard
Johannes Moll, Louisa Fay, Asfandyar Azhar, Sophie Ostmeier, Sergios Gatidis, Tim C. Lueth, Curtis Langlotz, Jean-Benoit Delbrouck
| Challenge: | Radiology reports lack a standardized format, limiting both interpretability and machine learning applications. |
| Approach: | They propose to use lightweight encoder-decoder models for structuring radiology reports . they compare models with eight open-source LLMs with prompting and in-context learning . |
| Outcome: | The proposed models outperform eight open-source LLMs on a human-annotated test set. |
RA-RRG: Multimodal Retrieval-Augmented Radiology Report Generation with Key Phrase Extraction (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing MLLMs are computationally expensive and may produce hallucinated content . RA-RRG uses large language models to generate radiology reports . |
| Approach: | They propose a retrieval-augmented RRG framework that combines multimodal retrieval with large language models to generate radiology reports. |
| Outcome: | RA-RRG uses large language models to generate radiology reports . it suppresses hallucinations while maintaining strong report generation performance . |
CheXalign: Preference fine-tuning in chest X-ray interpretation models without human feedback (2025.acl-long)
Copied to clipboard
Dennis Hein, Zhihong Chen, Sophie Ostmeier, Justin Xu, Maya Varma, Eduardo Pontes Reis, Arne Edward Michalson Md, Christian Bluethgen, Hyun Joo Shin, Curtis Langlotz, Akshay S Chaudhari
| Challenge: | Radiologists are a crucial role in translating medical images into actionable reports . however, the field faces staffing shortages and increasing workloads . |
| Approach: | They propose an automated pipeline for preference feedback focusing on chest X-ray radiology report generation (RRG) method leverages publicly available datasets containing pairs of images and radiologist-written reference reports with reference-based metrics, or Judges. |
| Outcome: | The proposed pipeline achieves state-of-the-art CheXbert scores on the MIMIC-CXR dataset while on average maintaining robust performance across six additional image perception and reasoning tasks. |
Multimodal Generation of Radiology Reports using Knowledge-Grounded Extraction of Entities and Relations (2022.aacl-main)
Copied to clipboard
Francesco Dalla Serra, William Clackett, Hamish MacKinnon, Chaoyang Wang, Fani Deligianni, Jeff Dalton, Alison Q. O’Neil
| Challenge: | Existing approaches to generate text radiology reports are prone to errors and poor clinical accuracy. |
| Approach: | They propose a two-step pipeline that subdivides the problem into factual triple extraction followed by free-text report generation. |
| Outcome: | The proposed pipeline shows that the generated reports exhibit realistic style but lack clinical accuracy. |
Multimodal Dual-Path Decoding for Medical Report Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Current methods for radiology report generation rely on encoder-decoder based frameworks that fail to integrate multimodal clinical evidence with domain-specific knowledge. |
| Approach: | They propose a multimodal dual-path framework that synergistically integrates large vision-language models and large language models for radiology report generation. |
| Outcome: | The proposed framework improves on the public MIMIC-CXR benchmark and shows that it is superior to state-of-the-art models. |
Extracting and Encoding: Leveraging Large Language Models and Medical Knowledge to Enhance Radiological Text Representation (2024.findings-acl)
Copied to clipboard
| Challenge: | Advancing representation learning in specialized fields like medicine remains challenging due to the scarcity of expert annotations for text and images. |
| Approach: | They propose a Fact Extractor that leverages large language models to extract factual statements from radiology reports. |
| Outcome: | The proposed framework outperforms current state-of-the-art methods in sentence ranking, natural language inference, and label extraction tasks. |
Controllable Chest X-Ray Report Generation from Longitudinal Representations (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Radiology reports are detailed text descriptions of the content of medical scans. |
| Approach: | They propose a method to align, concatenate and fuse the current and prior visual information into a joint longitudinal representation which can be provided to a multimodal report generation model. |
| Outcome: | The proposed method achieves state-of-the-art results while enabling anatomy-wise controllable report generation. |
Language over Labels: Contrastive Language Supervision Exceeds Purely Label-Supervised Classification Performance on Chest X-Rays (2022.aacl-srw)
Copied to clipboard
| Challenge: | Pretrained CLIP models lack domain-specific knowledge of text and images. |
| Approach: | They adapt CLIP-based models to the chest radiography domain using contrastive language supervision and a detailed ablation study of the batch and dataset size. |
| Outcome: | The proposed model outperforms supervised learning on labels on the MIMIC-CXR dataset while generalizing to the CheXpert and RSNA Pneumonia datasets. |
Improving Radiology Summarization with Radiograph and Anatomy Prompts (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies focus on automatic impression generation, but this task is time-consuming and in high demand. |
| Approach: | They propose to use an anatomy-enhanced multimodal model to generate automatic impressions by combining radiology images with textual features. |
| Outcome: | The proposed model achieves state-of-the-art on two benchmark datasets and compares with existing models. |