Papers by Sshubam Verma
Finding Blind Spots in Evaluator LLMs with Interpretable Checklists (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models are increasingly relied upon to evaluate text outputs of other LLMs . however, concerns persist over the accuracy of these assessments and the potential for misleading conclusions. |
| Approach: | They propose a framework to assess the reliability of Large Language Models (LLMs) they propose ' FBI' framework to examine the proficiency of Evaluator LLMs in assessing four critical abilities . |
| Outcome: | The proposed framework assesses the performance of LLMs in text generation tasks. |
MILU: A Multi-task Indic Language Understanding Benchmark (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing benchmarks focus on English, leaving substantial gaps in assessing LLM capabilities in low-resource and linguistically diverse languages. |
| Approach: | They propose a multi-task indic language understanding benchmark to assess LLMs in low-resource languages. |
| Outcome: | The new benchmark spans 8 domains and 41 subjects across 11 Indic languages, reflecting general and culturally specific knowledge. |