Papers by Hisham Cholakkal
Time Travel: A Comprehensive Benchmark to Evaluate LMMs on Historical and Cultural Artifacts (2025.findings-acl)
Copied to clipboard
Sara Ghaboura, Ketan Pravin More, Ritesh Thawkar, Wafa Al Ghallabi, Omkar Thawakar, Fahad Shahbaz Khan, Hisham Cholakkal, Salman Khan, Rao Muhammad Anwer
| Challenge: | TimeTravel is a benchmark of 10,250 expert-verified historical artifact samples spanning 266 distinct cultures across 10 major historical regions. |
| Approach: | They evaluate contemporary AI models on TimeTravel, highlighting their strengths and identifying areas for improvement. |
| Outcome: | The timeTravel benchmark covers 266 cultures and 10 major historical regions and aims to establish AI as reliable partner in preserving cultural heritage. |
LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMs (2025.findings-acl)
Copied to clipboard
Omkar Thawakar, Dinura Dissanayake, Ketan Pravin More, Ritesh Thawkar, Ahmed Heakl, Noor Ahsan, Yuhao Li, Ilmuz Zaman Mohammed Zumri, Jean Lahoud, Rao Muhammad Anwer, Hisham Cholakkal, Ivan Laptev, Mubarak Shah, Fahad Shahbaz Khan, Salman Khan
| Challenge: | Existing approaches do not emphasize step-wise problem-solving. |
| Approach: | They propose a visual reasoning chain benchmark and a fine-grained reasoning metric that evaluates correctness and logical coherence at each step. |
| Outcome: | The proposed framework outperforms existing models in six benchmarks and is 5x faster during inference scaling. |
Arabic Mini-ClimateGPT : A Climate Change and Sustainability Tailored Arabic LLM (2023.findings-emnlp)
Copied to clipboard
Sahal Mullappilly, Abdelrahman Shaker, Omkar Thawakar, Hisham Cholakkal, Rao Anwer, Salman Khan, Fahad Khan
| Challenge: | Recent large language models like ChatGPT and Bard excel in a wide variety of NLP tasks but are not specifically tailored for climate related domain specific information. |
| Approach: | They propose a lightweight Arabic Mini-ClimateGPT that is built on an open-source LLM and specifically fine-tuned on a conversational-style instruction tuning curated Arabic dataset Clima500-Instruct. |
| Outcome: | The proposed model surpasses the baseline LLM in 88.3% of cases during ChatGPT-based evaluation and human expert prefers it over other open-source models. |
MAviS: A Multimodal Conversational Assistant For Avian Species (2025.emnlp-main)
Copied to clipboard
Yevheniia Kryklyvets, Mohammed Irfan Kurpath, Sahal Shaji Mullappilly, Jinxing Zhou, Fahad Shahbaz Khan, Rao Muhammad Anwer, Salman Khan, Hisham Cholakkal
| Challenge: | Existing multimodal large language models face challenges when it comes to specialized topics like avian species. |
| Approach: | They propose a large-scale multimodal avian species dataset that integrates image, audio, and text modalities for over 1,000 bird species. |
| Outcome: | The proposed model outperforms the baseline MiniCPM-o-2.6 by a large margin. |
BiMediX2 : Bio-Medical EXpert LMM for Diverse Medical Modalities (2025.findings-emnlp)
Copied to clipboard
Sahal Shaji Mullappilly, Mohammed Irfan Kurpath, Sara Pieri, Saeed Yahya Alseiari, Shanavas Cholakkal, Khaled M Aldahmani, Fahad Shahbaz Khan, Rao Muhammad Anwer, Salman Khan, Timothy Baldwin, Hisham Cholakkal
| Challenge: | BiMediX2 is a bilingual (Arabic-English) large multimodal model that supports text-based and image-based medical interactions. |
| Approach: | They introduce BiMediX2, a bilingual (Arabic-English) Bio-Medical EXpert Large Multimodal Model that supports text-based and image-based medical interactions. |
| Outcome: | The model outperforms existing models by over 9% in English and more than 20% in Arabic evaluations. |
LLMVoX: Autoregressive Streaming Text-to-Speech Model for Any LLM (2025.findings-acl)
Copied to clipboard
Sambal Shikhar, Mohammed Irfan Kurpath, Sahal Shaji Mullappilly, Jean Lahoud, Fahad Shahbaz Khan, Rao Muhammad Anwer, Salman Khan, Hisham Cholakkal
| Challenge: | Existing speech-enabled LLMs degrade conversational quality by modifying the LLM, compromising its linguistic capabilities. |
| Approach: | They propose a lightweight 30M-parameter, LLM-agnostic, autoregressive streaming TTS system that generates high-quality speech with low latency. |
| Outcome: | The proposed system achieves a significantly lower word error rate compared to speech-enabled LLMs while operating at comparable latency. |
Paper Circle: An Open-source Multi-agent Research Discovery and Analysis Framework (2026.acl-long)
Copied to clipboard
| Challenge: | Recent advances in large language models have demonstrated strong potential for understanding user intent . paper describes system architecture, agent roles, retrieval and scoring methods, knowledge graph schema, and evaluation interfaces . |
| Approach: | They propose a multi-agent research discovery and analysis system that integrates multiple agents to reduce the effort required to find, assess, organize, and understand academic literature. |
| Outcome: | The proposed system reduces the effort required to find, assess, organize, and understand academic literature. |
Fann or Flop: A Multigenre, Multiera Benchmark for Arabic Poetry Understanding in LLMs (2025.emnlp-main)
Copied to clipboard
Wafa Al Ghallabi, Ritesh Thawkar, Sara Ghaboura, Ketan Pravin More, Omkar Thawakar, Hisham Cholakkal, Salman Khan, Rao Muhammad Anwer
| Challenge: | a benchmark is designed to assess the comprehension of Arabic poetry by large language models in 12 historical eras. |
| Approach: | They propose a benchmark to assess the comprehension of Arabic poetry by large language models in 12 historical eras. |
| Outcome: | The benchmark assesses the comprehension of Arabic poetry by large language models in 12 historical eras. |
BiMediX: Bilingual Medical Mixture of Experts LLM (2024.findings-emnlp)
Copied to clipboard
Sara Pieri, Sahal Shaji Mullappilly, Fahad Khan, Rao Anwer, Salman Khan, Timothy Baldwin, Hisham Cholakkal
| Challenge: | a new bilingual medical mixture of experts LLM is designed for seamless interaction in both English and Arabic. |
| Approach: | They propose a semi-automated English-to-Arabic translation pipeline with human refinement to ensure high-quality translations. |
| Outcome: | The proposed model outperforms state-of-the-art medical LLMs in Arabic and Arabic . it outperformed the generic Arabic-English bilingual LLM, Jais-30B by 10% and 15% . |
A Culturally-diverse Multilingual Multimodal Video Benchmark & Model (2025.emnlp-main)
Copied to clipboard
Bhuiyan Sanjid Shafique, Ashmal Vayani, Muhammad Maaz, Hanoona Abdul Rasheed, Dinura Dissanayake, Mohammed Irfan Kurpath, Yahya Hmaiti, Go Inoue, Jean Lahoud, Md. Safirur Rashid, Shadid Intisar Quasem, Maheen Fatima, Franco Vidal, Mykola Maslych, Ketan Pravin More, Sanoojan Baliah, Hasindri Watawana, Yuhao Li, Fabian Farestam, Leon Schaller, Roman Tymtsiv, Simon Weber, Hisham Cholakkal, Ivan Laptev, Shin’ichi Satoh, Michael Felsberg, Mubarak Shah, Salman Khan, Fahad Shahbaz Khan
| Challenge: | Large multimodal models have gained attention for their effectiveness to understand and generate descriptions of visual content. |
| Approach: | They propose a multilingual Video LMM benchmark to evaluate video LMMs across 14 languages . they also introduce a machine translated multilingual video training set . |
| Outcome: | The proposed video LMM benchmark is designed to evaluate video Lmms across 14 languages including Arabic, Bengali, Chinese, English, French, German, Hindi, Japanese, Russian, Sinhala, Spanish, Swedish, Tamil, and Urdu. |