Papers by Hammam Abdelwahab
Tokenizer Choice For LLM Training: Negligible or Crucial? (2024.findings-naacl)
Copied to clipboard
Mehdi Ali, Michael Fromm, Klaudia Thellmann, Richard Rutmann, Max Lübbering, Johannes Leveling, Katrin Klug, Jan Ebert, Niclas Doll, Jasper Buschhoff, Charvi Jain, Alexander Weber, Lena Jurkschat, Hammam Abdelwahab, Chelsea John, Pedro Ortiz Suarez, Malte Ostendorff, Samuel Weinbach, Rafet Sifa, Stefan Kesselheim, Nicolas Flores-Herr
| Challenge: | Recent success of large language models has been driven by curating the training dataset composition, scaling of model architectures and advancements in pretraining objectives, leaving tokenizer influence as a blind spot. |
| Approach: | They conduct a comprehensive study on the influence of tokenizer choice on LLM downstream performance by training 24 mono- and multilingual LLMs at a 2.6B parameter scale. |
| Outcome: | The proposed model can significantly impact the model's downstream performance and training costs. |
Can Continual Pretraining Bridge the Performance Gap between General-purpose and Specialized Language Models in the Medical Domain? (2026.acl-long)
Copied to clipboard
Niclas Doll, Jasper Schulze Buschhoff, Shalaka Satheesh, Hammam Abdelwahab, Héctor Allende-Cid, Katrin Klug
| Challenge: | specialized models have a large potential for translation and translation, but they lack the integration of domainspecific knowledge and terminology into clinical workflows. |
| Approach: | They construct a German medical corpus to continuously pre-train and merge three well-known LLMs and use it to improve model performance. |
| Outcome: | The proposed model family significantly outperforms the mistral-Small-24B-Instruct model family on German medical benchmarks. |