Papers by Sergei Tilga
JEEM: Vision-Language Understanding in Four Arabic Dialects (2026.findings-eacl)
Copied to clipboard
Karima Kadaoui, Hanin Atwany, Hamdan Al-Ali, Abdelrahman Mohamed, Ali Mekky, Sergei Tilga, Natalia Fedorova, Ekaterina Artemova, Hanan Aldarmaki, Yova Kementchedjhieva
| Challenge: | Existing evaluation datasets feature Western-centric images and English text, while their non-English counterparts are often derived from the latter. |
| Approach: | They propose to evaluate Vision-Language Models (VLMs) on visual understanding across four Arabic-speaking countries: Jordan, The Emirates, Egypt, and Morocco. |
| Outcome: | The proposed model underperforms in visual understanding and dialect-specific generation across four Arabic-speaking countries. |
Beemo: Benchmark of Expert-edited Machine-generated Outputs (2025.naacl-long)
Copied to clipboard
Ekaterina Artemova, Jason S Lucas, Saranya Venkatraman, Jooyoung Lee, Sergei Tilga, Adaku Uchendu, Vladislav Mikhailov
| Challenge: | Existing benchmarks for machine-generated texts (MGTs) include single-author texts (human-written and machine-generated). |
| Approach: | They propose to benchmark machine-generated outputs (Beemo) which includes 6.5k texts written by humans, generated by ten instruction-finetuned LLMs, and edited by experts for various use cases. |
| Outcome: | The proposed benchmark includes 6.5k texts written by humans, generated by ten instruction-finetuned LLMs, and edited by experts for various use cases, ranging from creative writing to summarization. |