Papers by Nishant Kumar
Aligning Complex Knowledge Graph Question Answering as Knowledge-Aware Constrained Code Generation (2025.coling-main)
Copied to clipboard
| Challenge: | Existing frameworks that generate LF using Large Language Models (LLMs) in a few-shot setting are limited due to little exposure to the LF during pre-training. |
| Approach: | They propose a framework that aligns the LF generation as code generation that incorporates LF-specific constraints. |
| Outcome: | The proposed framework surpasses all few-shot baselines on KQA Pro by 21%. |
DRISHTIKON: A Multimodal Multilingual Benchmark for Testing Language Models’ Understanding on Indian Culture (2025.emnlp-main)
Copied to clipboard
Arijit Maji, Raghvendra Kumar, Akash Ghosh, null Anushka, Nemil Shah, Abhilekh Borah, Vanshika Shah, Nishant Mishra, Sriparna Saha
| Challenge: | DRISHTIKON is a first-of-its-kind multimodal and multilingual benchmark centered exclusively on Indian culture. |
| Approach: | They evaluate a wide range of vision-language models across zero-shot and chain-of-thought settings and use them to evaluate cultural understanding of generative AI systems. |
| Outcome: | The DRISHTIKON dataset covers 15 languages, all states and union territories, and incorporating over 64,000 aligned text-image pairs. |
SymKGQA: Few-Shot Knowledge Graph Question Answering via Symbolic Program Generation and Execution (2024.acl-long)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models have led to low-level LFs that are limited to the knowledge of underlying LLM about the LF. |
| Approach: | They propose a framework that generates a symbolic LF in a few-shot setting using Large Language Models. |
| Outcome: | The proposed framework outperforms all other few-shot and many fully-supervised KGQA approaches. |
Let’s Play Across Cultures: A Large Multilingual, Multicultural Benchmark for Assessing Language Models’ Understanding of Sports (2025.emnlp-main)
Copied to clipboard
Punit Kumar Singh, Nishant Kumar, Akash Ghosh, Kunal Pasad, Khushi Soni, Manisha Jaishwal, Sriparna Saha, Syukron Abu Ishaq Alfarozi, Asres Temam Abagissa, Kitsuchart Pasupa, Haiqin Yang, Jose G Moreno
| Challenge: | Language Models (LMs) are primarily evaluated on globally popular sports, often overlooking regional and indigenous sporting traditions. |
| Approach: | They propose to use multiple-choice questions (MCQs) to assess LMs' understanding of traditional sports across 60 countries and 6 continents. |
| Outcome: | The new benchmark will be publicly available, fostering research in culturally aware AI systems. |