Papers by Amin Dada
On the Impact of Cross-Domain Data on German Language Models (2023.findings-emnlp)
Copied to clipboard
Amin Dada, Aokun Chen, Cheng Peng, Kaleb Smith, Ahmad Idrissi-Yaghir, Constantin Seibold, Jianning Li, Lars Heiliger, Christoph Friedrich, Daniel Truhn, Jan Egger, Jiang Bian, Jens Kleesiek, Yonghui Wu
| Challenge: | Traditionally, large language models have been trained on general web crawls or domain-specific data. |
| Approach: | They present a German dataset and a dataset aimed at containing high-quality data to examine the importance of data diversity over quality. |
| Outcome: | The proposed model outperforms models trained on quality data on multiple downstream tasks. |
A Modular Approach for Clinical SLMs Driven by Synthetic Data with Pre-Instruction Tuning, Model Merging, and Clinical-Tasks Alignment (2025.acl-long)
Copied to clipboard
Jean-Philippe Corbeil, Amin Dada, Jean-Michel Attendu, Asma Ben Abacha, Alessandro Sordoni, Lucas Caccia, Francois Beaulieu, Thomas Lin, Jens Kleesiek, Paul Vozila
| Challenge: | Large language models such as GPT-4 have limited their deployment in clinical settings . a novel framework for adapting SLMs into high-performing clinical models is needed . |
| Approach: | They propose a framework for adapting large language models into high-performing clinical models . they pre-instruct experts on relevant medical and clinical corpora and model merging . |
| Outcome: | The proposed framework outperforms the existing model on the CLUE+ benchmark on medical entities and radiology reports. |
Towards Conditioning Clinical Text Generation for User Control (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models exhibit hallucinations and factual inconsistencies necessitating human oversight. |
| Approach: | They propose to use Large Language Models as human proxies to condition LLMs for clinician control without increasing cognitive workload. |
| Outcome: | The proposed approach yields 9% relative improvement without augmented training and up to 34% with dataset augmentation. |
Comprehensive Study on German Language Models for Clinical and Biomedical Text Understanding (2024.lrec-main)
Copied to clipboard
Ahmad Idrissi-Yaghir, Amin Dada, Henning Schäfer, Kamyar Arzideh, Giulia Baldini, Jan Trienes, Max Hasin, Jeanette Bewersdorff, Cynthia S. Schmidt, Marie Bauer, Kaleb E. Smith, Jiang Bian, Yonghui Wu, Jörg Schlötterer, Torsten Zesch, Peter A. Horn, Christin Seifert, Felix Nensa, Jens Kleesiek, Christoph M. Friedrich
| Challenge: | Pre-trained language models can struggle in specialized domains such as medicine . existing generalpurpose pre-tried models can be used and refined through further pre-training on domainspecific unlabeled data. |
| Approach: | They pre-trained German medical language models on 2.4B tokens from translated public data and 3B token of German clinical data. |
| Outcome: | The proposed models outperform clinical models on various downstream tasks in germany . the authors show that continuous pre-training can match or exceed clinical models trained from scratch . |