Papers by Mostafa Masoudi
Advancing Persian LLM Evaluation (2025.findings-naacl)
Copied to clipboard
Sara Bourbour Hosseinbeigi, Behnam Rohani, Mostafa Masoudi, Mehrnoush Shamsfard, Zahra Saaberi, Mostafa Karimi Manesh, Mohammad Amin Abbasi
| Challenge: | Existing evaluation approaches for large language models in low-resource languages like Persian lack comprehensive frameworks, limiting their ability to assess models’ performance over a wide range of tasks requiring considerable cultural and contextual knowledge. |
| Approach: | They propose to provide two new benchmarks to assess models' performance over a wide range of tasks requiring considerable cultural and contextual knowledge. |
| Outcome: | The proposed benchmarks challenge the current state-of-the-art models’ abilities in a variety of Persian language comprehension tasks while reducing data contamination while providing an accurate assessment of Persian LLMs. |
Comparative Study of Multilingual Idioms and Similes in Large Language Models (2025.coling-main)
Copied to clipboard
Paria Khoshtab, Danial Namazifard, Mostafa Masoudi, Ali Akhgary, Samin Mahdizadeh Sani, Yadollah Yaghoobzadeh
| Challenge: | figurative language is one of the most challenging aspects of human language for LLMs to comprehend . |
| Approach: | They evaluate LLMs using two multilingual datasets on simile and idiom interpretation and two new evaluation sets for Persian . they find prompt engineering methods are generally effective, but their success varies by figurative type, language, and model. |
| Outcome: | The proposed models perform better in simile and idiom interpretations across languages and figurative types. |
Benchmarking Large Language Models for Persian: A Preliminary Study Focusing on ChatGPT (2024.lrec-main)
Copied to clipboard
Amirhossein Abaskohi, Sara Baruni, Mostafa Masoudi, Nesa Abbasi, Mohammad Hadi Babalou, Ali Edalat, Sepehr Kamahi, Samin Mahdizadeh Sani, Nikoo Naghavian, Danial Namazifard, Pouya Sadeghi, Yadollah Yaghoobzadeh
| Challenge: | a new study examines the efficacy of large language models (LLMs) for Persian . ChatGPT and LLMs have shown remarkable performance in English, but their efficiency for low-resource languages remains an open question. |
| Approach: | They present a benchmarking study of large language models (LLMs) for Persian . they focus on GPT-3.5-turbo, but also GPT-4 and OpenChat-3.5 . |
| Outcome: | The proposed model performs better in Persian than other low-resource languages . the study is the first comprehensive benchmarking of large language models . |
Matina: A Culturally-Aligned Persian Language Model Using Multiple LoRA Experts (2025.findings-acl)
Copied to clipboard
Sara Bourbour Hosseinbeigi, MohammadAli SeifKashani, Javad Seraj, Fatemeh Taherinezhad, Ali Nafisi, Fatemeh Nadi, Iman Barati, Hosein Hasani, Mostafa Amiri, Mostafa Masoudi
| Challenge: | Existing Large language models fail to accurately model underrepresented languages and cultures, limiting their applicability and acceptance. |
| Approach: | They develop a Persian-focused multi-expert model that incorporates Iranian cultural values and linguistic structures. |
| Outcome: | The proposed model outperforms baseline models in task performance and user satisfaction. |