Papers by Amr Mohamed
Beyond Random Sampling: Efficient Language Model Pretraining via Curriculum Learning (2026.eacl-long)
Copied to clipboard
| Challenge: | Curriculum learning has improved efficiency across machine learning domains, but remains underexplored for language model pretraining. |
| Approach: | They present a systematic investigation of curriculum learning in LLM pretraining . they use vanilla curriculum learning, pacing-based sampling, and interleaved curricula . |
| Outcome: | The proposed framework accelerates convergence in early and mid-training phases, reducing training steps by 18-45% to reach baseline performance. |
Lost in the Mix: Evaluating LLM Understanding of Code-Switched Text (2026.acl-long)
Copied to clipboard
| Challenge: | Code-switching (CSW) is widespread in multilingual communities and increasingly prevalent in online content. |
| Approach: | They propose a pipeline for producing linguistically grounded CSW variants of established benchmarks across five typologically diverse languages. |
| Outcome: | The proposed model sets show that inserting non-English tokens into English reduces accuracy on comprehension and reasoning benchmarks, whereas embedding English into non- English contexts often improves it. |
LLM as a Broken Telephone: Iterative Generation Distorts Information (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models are increasingly responsible for online content, but they can be distorted by repeated transmission. |
| Approach: | They investigate whether large language models distort information through iterative generation. |
| Outcome: | The findings raise important questions about the reliability of LLM-generated content in iterative workflows. |
Domain Specific Sub-network for Multi-Domain Neural Machine Translation (2022.aacl-short)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) is based on transformer models that are trained on general data from a single language pair or multiple languages. |
| Approach: | They propose a method to make masks unique per domain to improve generalization to unseen domains. |
| Outcome: | The proposed method outperforms continue training on multi-domain data on German to English translation by 1.47 BLEU points and on new domains by 1.52 BLUE points. |
Fast-Decoding Diffusion Language Models via Progress-Aware Confidence Schedules (2026.findings-acl)
Copied to clipboard
| Challenge: | *SchED* is a training-free, model-agnostic early-exit algorithm that terminates diffusion decoding using a progress-aware confidence threshold. |
| Approach: | They propose a training-free, model-agnostic early-exit algorithm that terminates diffusion decoding using a progress-aware confidence threshold. |
| Outcome: | The proposed algorithm achieves 4 speedups on instruction-tuned models while maintaining baseline performance on average. |
Cultural Benchmarking of LLMs in Standard and Dialectal Arabic Dialogues (2026.acl-long)
Copied to clipboard
Muhammad Dehan Al Kautsar, Saeed Almheiri, Momina Ahsan, Bilal Elbouardi, Younes Samih, Sarfraz Ahmad, Amr Keleg, Omar El Herraoui, Kareem Elzeky, Abed Alhakim Freihat, Mohamed Anwar, Zhuohan Xie, Junhong Liang, Mohammad Rustom Al Nasar, Preslav Nakov, Fajri Koto
| Challenge: | Most benchmarks focus on short text snippets in Modern Standard Arabic (MSA), overlooking cultural nuances that naturally arise in dialogues. |
| Approach: | They propose a culturally grounded conversational dataset covering 13 Arabic-speaking countries, in both Modern Standard Arabic (MSA) and each country’s respective dialect, spanning 12 daily-life topics and 54 fine-grained subtopics. |
| Outcome: | The proposed model performs worse on all three tasks than the MSA benchmark. |