Papers by Herumb Shandilya
Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning (2024.acl-long)
Copied to clipboard
Shivalika Singh, Freddie Vargus, Daniel D’souza, Börje Karlsson, Abinaya Mahendiran, Wei-Yin Ko, Herumb Shandilya, Jay Patel, Deividas Mataciunas, Laura O’Mahony, Mike Zhang, Ramith Hettiarachchi, Joseph Wilson, Marina Machado, Luisa Moura, Dominik Krzemiński, Hakimeh Fadaei, Irem Ergun, Ifeoma Okoh, Aisha Alaagib, Oshan Mudannayake, Zaid Alyafeai, Vu Chien, Sebastian Ruder, Surya Guthikonda, Emad Alghamdi, Sebastian Gehrmann, Niklas Muennighoff, Max Bartolo, Julia Kreutzer, Ahmet Üstün, Marzieh Fadaee, Sara Hooker
| Challenge: | Existing datasets in the English language are mostly in the realm of instruction fine-tuning . aya dataset, the Aya Collection, and the AYa Evaluation Suite are key resources . |
| Approach: | They aim to build a human-curated instruction-following dataset spanning 65 languages . they work with fluent speakers of languages from around the world to collect natural instances of instructions and completions . |
| Outcome: | The goal is to build a human-curated instruction-following dataset spanning 65 languages. |
Beyond a Single Extractor: Re-thinking HTML-to-Text Extraction for LLM Pre-training (2026.findings-eacl)
Copied to clipboard
Jeffrey Li, Joshua P Gardner, Doug Kang, Fangping Shi, Karanjeet Singh, Chun-Liang Li, Herumb Shandilya, David Leo Wright Hall, Oncel Tuzel, Percy Liang, Ludwig Schmidt, Hadi Pouransari, Fartash Faghri
| Challenge: | Existing open-source datasets predominantly apply a single fixed extractor to all webpages. |
| Approach: | They propose to take a Union over different extractors to improve model performance . they show that extractor choice can significantly impact downstream task performance based on content type . |
| Outcome: | The proposed approach can increase the token yield of DCLM-Baseline by 71% while maintaining benchmark performance. |