Can Medical Vision-Language Pre-training Succeed with Purely Synthetic Data? (2025.findings-acl)
Copied to clipboard
Che Liu, Zhongwei Wan, Haozhe Wang, Yinda Chen, Talha Qaiser, Chen Jin, Nikolay Burlutskiy, Fariba Yousefi, Rossella Arcucci
| Challenge: | Medical Vision-Language Pretraining (MedVLP) models typically require large-scale datasets with paired, high-quality image-text data. |
| Approach: | They propose to generate large-scale synthetic image-text pairs using off-the-shelf generative models . they propose to isolate model and training settings, focusing entirely from the data perspective. |
| Outcome: | The proposed pipeline outperforms models trained on real data by 3.8% on averaged AUC on zero-shot classification tasks. |
Similar Papers
Medical Adaptation of Large Language and Vision-Language Models: Are We Making Progress? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Several studies claim that domain-adaptive pretraining improves performance on downstream medical tasks. |
| Approach: | They compare medical LLMs and VLMs against their corresponding base models . they find that medical Lms outperform their base models in 12.1% of cases . |
| Outcome: | The proposed models outperform their base models on medical questions and tasks in 12.1% of cases and reach a tie in 49.8% of cases. |
Medical Vision-Language Pre-Training for Brain Abnormalities (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing vision-language models lack expertise for medical applications due to the scarcity and complexity of data. |
| Approach: | They propose a pipeline to collect medical image-text aligned data for pretraining from public resources such as PubMed and build a high-performance vision-language model tailored to specific medical tasks. |
| Outcome: | The proposed model is based on a large brain image-text dataset and will be released to the public. |
When Raw Data Prevails: Are Large Language Model Embeddings Effective in Numerical Data Representation for Medical Machine Learning Applications? (2024.findings-emnlp)
Copied to clipboard
Yanjun Gao, Skatje Myers, Shan Chen, Dmitriy Dligach, Timothy Miller, Danielle Bitterman, Matthew Churpek, Majid Afshar
| Challenge: | Numerical data is pivotal for medical questions and answers, but tabular data is not fully integrated into LLMs. |
| Approach: | They examine the effectiveness of vector representations from last hidden states of LLMs for medical diagnostics and prognostics using electronic health record data. |
| Outcome: | The proposed representations outperform those using raw numerical EHR data in medical diagnostics and prognostics. |
MediVLM: A Vision Language Model for Radiology Report Generation from Medical Images (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for radiology report generation from medical images are incomplete and inconsistent, fail to focus on informative regions within an image and impose strong annotation assumptions for model training. |
| Approach: | They propose a vision language model (VLM) for radiology report generation from medical images that uses a pre-trained object detector to extract the salient anatomical regions from images, an image encoder, a text encoder and a transformer based decoder to generate the final report. |
| Outcome: | The proposed model generates radiology reports even when no reports are available for training. |
Med-MoE: Mixture of Domain-Specific Experts for Lightweight Medical Vision-Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in multimodal large language models have seen remarkable progress for medical decision-making, however, they are designated for specific classification or generative tasks and require model training or finetuning on large-scale datasets with sizeable parameters and tremendous computing. |
| Approach: | They propose a framework that tackles discriminative and generative multimodal medical tasks using multimodal alignment, instruction tuning and routing. |
| Outcome: | The proposed model can achieve superior performance to or on par with state-of-the-art baselines while only requiring 30%-50% of activated model parameters. |
Demystifying Synthetic Data in LLM Pre-training: A Systematic Study of Scaling Laws, Benefits, and Pitfalls (2025.emnlp-main)
Copied to clipboard
Feiyang Kang, Newsha Ardalani, Michael Kuchnik, Youssef Emad, Mostafa Elhoushi, Shubhabrata Sengupta, Shang-Wen Li, Ramya Raghavendra, Ruoxi Jia, Carole-Jean Wu
| Challenge: | a large-scale empirical study compares natural web data, diverse synthetic types, and mixtures of natural and synthetic data. |
| Approach: | They conduct a large-scale empirical study on large-volume LLMs using a unified protocol and scaling laws. |
| Outcome: | The proposed method is faster than pre-training on natural web data, the authors show . their results are consistent with previous studies on rephrased text and textbooks . |
Vision-Language Pretraining: Current Trends and the Future (2022.acl-tutorials)
Copied to clipboard
| Challenge: | Recent vision-language models are being used for downstream tasks that require large datasets and supervised datasets. |
| Approach: | They focus on recent vision-language pretraining paradigms and their strengths and shortcomings . they compare the different family of models used for vision- language pretraining . |
| Outcome: | This paper provides the background on image–language datasets, benchmarks, and modeling innovations before the multimodal pretraining area. |
MMCLIP: Cross-Modal Attention Masked Modelling for Medical Language-Image Pre-Training (2026.acl-long)
Copied to clipboard
| Challenge: | Existing vision-and-language pretraining methods face challenges in reconstructing pathological features due to limited data. |
| Approach: | They propose a method that uses masked modeling to enhance visual and linguistic learning. |
| Outcome: | MMCLIP integrates unpaired data through disease-kind prompts to achieve state-of-the-art performance in zero-shot and fine-tuning across five benchmarks. |
Pre-trained Language Models Can be Fully Zero-Shot Learners (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to pre-trained language models require fine-tuning on labeled datasets or manually constructing proper prompts. |
| Approach: | They propose a nonparametric prompting PLM for fully zero-shot language understanding . they compare it to previous methods for text classification and text entailment . |
| Outcome: | The proposed method outperforms previous methods on diverse tasks. |
MediSwift: Efficient Sparse Pre-trained Biomedical Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models are typically trained on general source data forvarious domains, but domain-specific pre-training is expensive and requires computational costs. |
| Approach: | They propose a suite of biomedicalLMs that leverage sparse pre-training on domain-specific biomedically text data. |
| Outcome: | The proposed model outperforms existing LLMs on biomedical tasks by 22.5x . |