Papers by Shruti Rijhwani
SEAHORSE: A Multilingual, Multifaceted Dataset for Summarization Evaluation (2023.emnlp-main)
Copied to clipboard
Elizabeth Clark, Shruti Rijhwani, Sebastian Gehrmann, Joshua Maynez, Roee Aharoni, Vitaly Nikolaev, Thibault Sellam, Aditya Siddhant, Dipanjan Das, Ankur Parikh
| Challenge: | evaluating the quality of generated text is a difficult problem for large language models. |
| Approach: | They propose a dataset for multilingual, multifaceted summarization evaluation. |
| Outcome: | The proposed dataset can be used to train multilingual summarization systems . it shows that the dataset performs well on the out-of-domain meta-evaluation benchmarks TRUE and mFACE . |
MasakhaNER: Named Entity Recognition for African Languages (2021.tacl-1)
Copied to clipboard
David Ifeoluwa Adelani, Jade Abbott, Graham Neubig, Daniel D’souza, Julia Kreutzer, Constantine Lignos, Chester Palen-Michel, Happy Buzaaba, Shruti Rijhwani, Sebastian Ruder, Stephen Mayhew, Israel Abebe Azime, Shamsuddeen H. Muhammad, Chris Chinenye Emezue, Joyce Nakatumba-Nabende, Perez Ogayo, Aremu Anuoluwapo, Catherine Gitau, Derguene Mbaye, Jesujoba Alabi, Seid Muhie Yimam, Tajuddeen Rabiu Gwadabe, Ignatius Ezeani, Rubungo Andre Niyongabo, Jonathan Mukiibi, Verrah Otiende, Iroro Orife, Davis David, Samba Ngom, Tosin Adewumi, Paul Rayson, Mofetoluwa Adeyemi, Gerald Muriuki, Emmanuel Anebi, Chiamaka Chukwuneke, Nkiruka Odu, Eric Peter Wairagala, Samuel Oyerinde, Clemencia Siro, Tobius Saul Bateesa, Temilola Oloyede, Yvonne Wambui, Victor Akinode, Deborah Nabagereka, Maurice Katusiime, Ayodele Awokoya, Mouhamadane MBOUP, Dibora Gebreyohannes, Henok Tilaye, Kelechi Nwaike, Degaga Wolde, Abdoulaye Faye, Blessing Sibanda, Orevaoghene Ahia, Bonaventure F. P. Dossou, Kelechi Ogueji, Thierno Ibrahima DIOP, Abdoulaye Diallo, Adewale Akinfaderin, Tendai Marengereke, Salomey Osei
| Challenge: | (2020) African languages are underrepresented in existing natural language processing datasets, research, and tools due to lack of datasets and reproducible results. |
| Approach: | They propose to create a dataset for named entity recognition (NER) in ten African languages. |
| Outcome: | The results of the first large dataset for named entity recognition (NER) in ten African languages are released to inform future research on African NLP. |
Soft Gazetteers for Low-Resource Named Entity Recognition (2020.acl-main)
Copied to clipboard
| Challenge: | Existing named entity recognition models use gazetteers to improve performance, but they are limited in coverage and do not exist in low-resource languages. |
| Approach: | They propose a method that integrates Wikipedia information into named entity models by cross-lingual entity linking. |
| Outcome: | The proposed method improves on four low-resource languages with Wikipedia . it incorporates available information from english knowledge bases into neural models . |
OCR Post Correction for Endangered Language Texts (2020.emnlp-main)
Copied to clipboard
| Challenge: | Currently, there is little to no data available to build natural language processing models for endangered languages. |
| Approach: | They propose a benchmark dataset of transcriptions for scanned books in three critically endangered languages and a method to improve OCR in these data-scarce settings. |
| Outcome: | The proposed method reduces the recognition error rate by 34% across the three endangered languages. |
Evaluating the Morphosyntactic Well-formedness of Generated Texts (2021.emnlp-main)
Copied to clipboard
Adithya Pratapa, Antonios Anastasopoulos, Shruti Rijhwani, Aditi Chaudhary, David R. Mortensen, Graham Neubig, Yulia Tsvetkov
| Challenge: | Text generation systems are ubiquitous in natural language processing applications, but evaluation of these systems remains a challenge, especially in multilingual settings. |
| Approach: | They propose a metric to evaluate the morphosyntactic well-formedness of text using its dependency parse and morphologically-rich rules of the language. |
| Outcome: | The proposed metric can evaluate the morphosyntactic well-formedness of text using its dependency parse and morphologically-rich rules of the language. |
XTREME-UP: A User-Centric Scarce-Data Benchmark for Under-Represented Languages (2023.findings-emnlp)
Copied to clipboard
Sebastian Ruder, Jonathan Clark, Alexander Gutkin, Mihir Kale, Min Ma, Massimo Nicosia, Shruti Rijhwani, Parker Riley, Jean-Michel Sarr, Xinyi Wang, John Wieting, Nitish Gupta, Anna Katanova, Christo Kirov, Dana Dickinson, Brian Roark, Bidisha Samanta, Connie Tao, David Adelani, Vera Axelrod, Isaac Caswell, Colin Cherry, Dan Garrette, Reeve Ingle, Melvin Johnson, Dmitry Panteleev, Partha Talukdar
| Challenge: | Existing datasets are often informed by established research directions in the NLP community. |
| Approach: | They propose a benchmark to evaluate the capabilities of language models across 88 under-represented languages over 9 key user-centric technologies including ASR, OCR, MT, and information access tasks. |
| Outcome: | The proposed benchmark evaluates the capabilities of language models across 88 under-represented languages over 9 key user-centric technologies including ASR, OCR, MT, and information access tasks. |
Parser combinators for Tigrinya and Oromo morphology (L18-1)
Copied to clipboard
Patrick Littell, Tom McCoy, Na-Rae Han, Shruti Rijhwani, Zaid Sheikh, David Mortensen, Teruko Mitamura, Lori Levin
| Challenge: | morphological parsers for two Afroasiatic languages are developed using a parser-combinator paradigm . the paradigm allows rapid development and ease of integration with other systems, but at a cost of non-optimal theoretical efficiency. |
| Approach: | They propose a rule-based morphological parser paradigm for Tigrinya and Oromo languages . they use a parsers-combinator paradigm instead of a finite-state paradigm . |
| Outcome: | The proposed paradigm allows rapid development and ease of integration with other systems, but at cost of non-optimal theoretical efficiency. |
Improving Candidate Generation for Low-resource Cross-lingual Entity Linking (2020.tacl-1)
Copied to clipboard
| Challenge: | Existing approaches to cross-lingual entity linking (XEL) do not extend well to low-resource languages with few Wikipedia pages. |
| Approach: | They propose to improve the model by combining Wikipedia references with a list of plausible candidate entities. |
| Outcome: | The proposed method yields 16.9% in Top-30 gold candidate recall compared with state-of-the-art models. |
WMT24++: Expanding the Language Coverage of WMT24 to 55 Languages & Dialects (2025.findings-acl)
Copied to clipboard
Daniel Deutsch, Eleftheria Briakou, Isaac Rayburn Caswell, Mara Finkelstein, Rebecca Galor, Juraj Juraska, Geza Kovacs, Alison Lui, Ricardo Rei, Jason Riesa, Shruti Rijhwani, Parker Riley, Elizabeth Salesky, Firas Trabelsi, Stephanie Winkler, Biao Zhang, Markus Freitag
| Challenge: | In order to evaluate large language models (LLMs), it is important to collect benchmark datasets in order to assess their multilingual performance. |
| Approach: | They extend the WMT24 dataset to cover 55 languages by collecting new human-written references and post-edits for 46 new languages/dialects. |
| Outcome: | The proposed dataset covers 55 languages and provides best-performing MT systems in all 55 languages. |
Towards Zero-resource Cross-lingual Entity Linking (D19-61)
Copied to clipboard
| Challenge: | XEL is challenging for most languages because of limited availability of requisite resources . simulated environments that use significant resources are not available in truly low-resource languages . |
| Approach: | They propose improvements to entity candidate generation and disambiguation to make better use of the limited resources available in low-resource languages. |
| Outcome: | The proposed model gains 6-20% end-to-end linking accuracy on four low-resource languages. |
Temporally-Informed Analysis of Named Entity Recognition (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods to evaluate text data are rarely reported by taking the timestamp of the document into account. |
| Approach: | They propose methods that make better use of temporally-diverse training data with a focus on named entity recognition. |
| Outcome: | The proposed models make better use of temporally-diverse training data, with a focus on named entity recognition. |
MasakhaNER 2.0: Africa-centric Transfer Learning for Named Entity Recognition (2022.emnlp-main)
Copied to clipboard
David Adelani, Graham Neubig, Sebastian Ruder, Shruti Rijhwani, Michael Beukman, Chester Palen-Michel, Constantine Lignos, Jesujoba Alabi, Shamsuddeen Muhammad, Peter Nabende, Cheikh M. Bamba Dione, Andiswa Bukula, Rooweither Mabuya, Bonaventure F. P. Dossou, Blessing Sibanda, Happy Buzaaba, Jonathan Mukiibi, Godson Kalipe, Derguene Mbaye, Amelia Taylor, Fatoumata Kabore, Chris Chinenye Emezue, Anuoluwapo Aremu, Perez Ogayo, Catherine Gitau, Edwin Munkoh-Buabeng, Victoire Memdjokam Koagne, Allahsera Auguste Tapo, Tebogo Macucwa, Vukosi Marivate, Mboning Tchiaze Elvis, Tajuddeen Gwadabe, Tosin Adewumi, Orevaoghene Ahia, Joyce Nakatumba-Nabende, Neo Lerato Mokono, Ignatius Ezeani, Chiamaka Chukwuneke, Mofetoluwa Oluwaseun Adeyemi, Gilles Quentin Hacheme, Idris Abdulmumin, Odunayo Ogundepo, Oreen Yousuf, Tatiana Moteu, Dietrich Klakow
| Challenge: | Existing studies on named entity recognition methods for African languages focus on English as the source language, but there is evidence that it is not the best for low-resource languages. |
| Approach: | They propose to use human-annotated datasets to analyze named entity recognition tasks in 20 African languages to test whether they are effective. |
| Outcome: | The proposed method improves zero-shot F1 scores by 14% over 20 languages compared to using English . |
Lexically Aware Semi-Supervised Learning for OCR Post-Correction (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing methods for digitizing text in endangered languages rely on manual data curated by the user. |
| Approach: | They propose a semi-supervised learning method that utilizes raw images to improve performance. |
| Outcome: | The proposed method reduces errors by 15%–29% on four endangered languages. |
Choosing Transfer Languages for Cross-Lingual Learning (P19-1)
Copied to clipboard
Yu-Hsiang Lin, Chian-Yu Chen, Jean Lee, Zirui Li, Yuyan Zhang, Mengzhou Xia, Shruti Rijhwani, Junxian He, Zhisong Zhang, Xuezhe Ma, Antonios Anastasopoulos, Patrick Littell, Graham Neubig
| Challenge: | Cross-lingual transfer is a useful tool for improving performance of natural language processing (NLP) on low-resource languages. |
| Approach: | They propose to use cross-lingual transfer to improve accuracy of low-resource languages . they build models that consider features to perform prediction on such languages based on ranking problem . |
| Outcome: | The proposed model predicts good transfer languages much better than baselines considering single features in isolation. |
AlloVera: A Multilingual Allophone Database (2020.lrec-1)
Copied to clipboard
David R. Mortensen, Xinjian Li, Patrick Littell, Alexis Michaud, Shruti Rijhwani, Antonios Anastasopoulos, Alan W Black, Florian Metze, Graham Neubig
| Challenge: | Phonemes are contrastive phonological units, and allophones are their various concrete realizations. |
| Approach: | They propose a resource that maps allophones to phonemes for 14 languages . they propose phonological representations that are much closer to a universal transcription . |
| Outcome: | The proposed resource maps from 218 allophones to phonemes for 14 languages. |