MediVLM: A Vision Language Model for Radiology Report Generation from Medical Images (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for radiology report generation from medical images are incomplete and inconsistent, fail to focus on informative regions within an image and impose strong annotation assumptions for model training. |
| Approach: | They propose a vision language model (VLM) for radiology report generation from medical images that uses a pre-trained object detector to extract the salient anatomical regions from images, an image encoder, a text encoder and a transformer based decoder to generate the final report. |
| Outcome: | The proposed model generates radiology reports even when no reports are available for training. |
Similar Papers
Fine-grained Medical Vision-Language Representation Learning for Radiology Report Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to learn medical vision-language representations by contrasting images with entire reports are not effective. |
| Approach: | They propose a phenotype-driven medical vision-language representation learning framework to bridge the gap between visual and textual modalities for improved text-oriented generation. |
| Outcome: | The proposed framework bridges the gap between visual and textual modalities for improved radiology report generation. |
Controllable Chest X-Ray Report Generation from Longitudinal Representations (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Radiology reports are detailed text descriptions of the content of medical scans. |
| Approach: | They propose a method to align, concatenate and fuse the current and prior visual information into a joint longitudinal representation which can be provided to a multimodal report generation model. |
| Outcome: | The proposed method achieves state-of-the-art results while enabling anatomy-wise controllable report generation. |
Extracting and Encoding: Leveraging Large Language Models and Medical Knowledge to Enhance Radiological Text Representation (2024.findings-acl)
Copied to clipboard
| Challenge: | Advancing representation learning in specialized fields like medicine remains challenging due to the scarcity of expert annotations for text and images. |
| Approach: | They propose a Fact Extractor that leverages large language models to extract factual statements from radiology reports. |
| Outcome: | The proposed framework outperforms current state-of-the-art methods in sentence ranking, natural language inference, and label extraction tasks. |
Can Medical Vision-Language Pre-training Succeed with Purely Synthetic Data? (2025.findings-acl)
Copied to clipboard
Che Liu, Zhongwei Wan, Haozhe Wang, Yinda Chen, Talha Qaiser, Chen Jin, Nikolay Burlutskiy, Fariba Yousefi, Rossella Arcucci
| Challenge: | Medical Vision-Language Pretraining (MedVLP) models typically require large-scale datasets with paired, high-quality image-text data. |
| Approach: | They propose to generate large-scale synthetic image-text pairs using off-the-shelf generative models . they propose to isolate model and training settings, focusing entirely from the data perspective. |
| Outcome: | The proposed pipeline outperforms models trained on real data by 3.8% on averaged AUC on zero-shot classification tasks. |
Multimodal Generation of Radiology Reports using Knowledge-Grounded Extraction of Entities and Relations (2022.aacl-main)
Copied to clipboard
Francesco Dalla Serra, William Clackett, Hamish MacKinnon, Chaoyang Wang, Fani Deligianni, Jeff Dalton, Alison Q. O’Neil
| Challenge: | Existing approaches to generate text radiology reports are prone to errors and poor clinical accuracy. |
| Approach: | They propose a two-step pipeline that subdivides the problem into factual triple extraction followed by free-text report generation. |
| Outcome: | The proposed pipeline shows that the generated reports exhibit realistic style but lack clinical accuracy. |
Automated Generation of Accurate & Fluent Medical X-ray Reports (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing medical report generation efforts focus on producing human-readable reports, yet the generated text may not be well aligned to the clinical facts. |
| Approach: | They propose to automate the generation of medical reports from chest X-ray image inputs . medical reports are the primary medium, which physicians communicate findings from scans - authors say . |
| Outcome: | The proposed method achieves fluency and clinical accuracy on common metrics. |
Pushing the Limits of Radiology with Joint Modeling of Visual and Textual Information (P18-3)
Copied to clipboard
| Challenge: | Recent research has focused on the intersection of computer vision and natural language processing, but its adaption to the medical domain is not fully explored. |
| Approach: | They aim to develop machine learning models that can reason jointly on medical images and clinical text for advanced search, retrieval, annotation and description of medical images. |
| Outcome: | The proposed models can reason jointly on medical images and clinical text for advanced search, retrieval, annotation and description of medical images. |
Enhancing Image-to-Text Generation in Radiology Reports through Cross-modal Multi-Task Learning (2024.lrec-main)
Copied to clipboard
| Challenge: | Image-to-text generation relies on independent models for image understanding and natural language generation, which often exhibit a semantic gap between visual and textual information. |
| Approach: | They propose a multi-task learning framework to leverage both visual and non-imaging data for generating radiology reports. |
| Outcome: | The proposed framework improves performance over single-task baselines across language generation metrics and mitigates overfitting in auxiliary tasks. |
Argus: Benchmarking and Enhancing Vision-Language Models for 3D Radiology Report Generation (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing work on 3D radiograph report generation focuses on 2D images, but 3D medical images provide more comprehensive diagnostic information. |
| Approach: | They propose a comprehensive training recipe for building high-performing VLMs for 3DRRG using a publicly available 3D CT-report dataset. |
| Outcome: | The proposed model achieves superior performance across different model sizes and input 3D medical image resolutions. |
EchoVLM: Dynamic Mixture-of-Experts Vision-Language Model for Universal Ultrasound Intelligence (2026.acl-long)
Copied to clipboard
| Challenge: | Ultrasound is the preferred early cancer screening modality due to non-ionizing radiation, cost-effectiveness, and real-time imaging. |
| Approach: | They propose to use ultrasound-tailored vision-language models with a mixture-of-experts architecture to train ultrasound-specific knowledge across seven anatomical systems. |
| Outcome: | The proposed model outperforms Qwen2-VL by 7.58 BLEU-1 and 3.45 ROUGE-1 points in report generation. |