Papers by Abhay Gupta
NovelHopQA: Diagnosing Multi-Hop Reasoning Failures in Long Narrative Contexts (2025.emnlp-main)
Copied to clipboard
| Challenge: | Current large language models struggle to answer questions that span tens of thousands of tokens. |
| Approach: | They evaluate 1–4 hop QA over 64k–128k-token excerpts from 83 novels . they find consistent accuracy drops with increased hops and context length . |
| Outcome: | The novelhopqa benchmark evaluates 1–4 hop QA over 64k–128k-token excerpts from 83 public-domain novels. |
EnDive: A Cross-Dialect Benchmark for Fairness and Performance in Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing benchmarks often overlook intra-language variations, leaving speakers of non-standard dialects underserved. |
| Approach: | EnDive evaluates seven state-of-the-art large language models across tasks . human evaluations confirm high translation quality, with average scores of at least 6.02/7 . |
| Outcome: | EnDive evaluates state-of-the-art large language models across language understanding, reasoning, mathematics, logic tasks. |