Papers by Sumanth Doddapaneni
Finding Blind Spots in Evaluator LLMs with Interpretable Checklists (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models are increasingly relied upon to evaluate text outputs of other LLMs . however, concerns persist over the accuracy of these assessments and the potential for misleading conclusions. |
| Approach: | They propose a framework to assess the reliability of Large Language Models (LLMs) they propose ' FBI' framework to examine the proficiency of Evaluator LLMs in assessing four critical abilities . |
| Outcome: | The proposed framework assesses the performance of LLMs in text generation tasks. |
Naamapadam: A Large-Scale Named Entity Annotated Data for Indic Languages (2023.acl-long)
Copied to clipboard
Arnav Mhaske, Harshit Kedia, Sumanth Doddapaneni, Mitesh M. Khapra, Pratyush Kumar, Rudra Murthy, Anoop Kunchukuttan
| Challenge: | Named Entity Recognition (NER) is a fundamental task in natural language processing (NLP). |
| Approach: | They present the largest publicly available Named Entity Recognition dataset for the 11 major Indian languages from two language families. |
| Outcome: | The proposed dataset is the largest publicly available Named Entity Recognition (NER) dataset for the 11 major Indian languages from two language families. |
Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languages (2022.tacl-1)
Copied to clipboard
Gowtham Ramesh, Sumanth Doddapaneni, Aravinth Bheemaraj, Mayank Jobanputra, Raghavan AK, Ajitesh Sharma, Sujit Sahoo, Harshita Diddee, Mahalakshmi J, Divyanshu Kakwani, Navneet Kumar, Aswin Pradeep, Srihari Nagaraj, Kumar Deepak, Vivek Raghavan, Anoop Kunchukuttan, Pratyush Kumar, Mitesh Shantadevi Khapra
| Challenge: | We present Samanantar, the largest publicly available parallel corpora collection for Indic languages . based on existing corporative, there has been limited benefit for resource-poor languages despite the lack of parallel corporals and monolingual corporata. |
| Approach: | They compile 12.4 million sentence pairs from existing corpora and mine 37.4 million from the Web. |
| Outcome: | The proposed model outperforms existing models and benchmarks on public datasets. |
Towards Leaving No Indic Language Behind: Building Monolingual Corpora, Benchmark and Models for Indic Languages (2023.acl-long)
Copied to clipboard
Sumanth Doddapaneni, Rahul Aralikatte, Gowtham Ramesh, Shreya Goyal, Mitesh M. Khapra, Anoop Kunchukuttan, Pratyush Kumar
| Challenge: | Recent advances in Natural Language Understanding are driven by pretrained multilingual models, which can potentially reduce the performance gap between high-resource languages through zero-shot knowledge transfer. |
| Approach: | They propose to create a human-supervised benchmark for Indic languages, IndicXTREME, with nine diverse NLU tasks covering 20 languages. |
| Outcome: | The proposed model improves on the monolingual corpora, IndicCorp, and IndicBERT in Indic languages with 105 evaluation sets across languages and tasks. |
Varta: A Large-Scale Headline-Generation Dataset for Indic Languages (2023.findings-acl)
Copied to clipboard
| Challenge: | Varta dataset includes more than 41 million pairs of headlines and articles in 14 different Indic languages (and English) |
| Approach: | They present a large-scale multilingual dataset for headline generation in Indic languages. |
| Outcome: | The Varta dataset includes more than 41 million pairs of headlines and articles in 14 different Indic languages (and English) the data can be used to train strong language models that outperform competitive baselines in both NLU and NLG benchmarks. |
Cross-Lingual Auto Evaluation for Assessing Multilingual LLMs (2025.acl-long)
Copied to clipboard
Sumanth Doddapaneni, Mohammed Safi Ur Rahman Khan, Dilip Venkatesh, Raj Dabre, Anoop Kunchukuttan, Mitesh M Khapra
| Challenge: | Evaluating machine-generated text remains a challenge in NLP for non-English languages . current evaluation frameworks focus on English, revealing a gap in multilingual evaluations . |
| Approach: | They propose a cross-lingual auto evaluation framework that includes evaluator LLMs and a test set specifically designed for multilingual evaluation. |
| Outcome: | The proposed model aligns more closely with human judgments than proprietary models on non-English language evaluations. |