Papers by Nikhil Singh
Universal Adversarial Triggers for Attacking and Analyzing NLP (D19-1)
Copied to clipboard
| Challenge: | Using adversarial triggers, a model can produce a specific prediction . adversarial attacks are useful for evaluation and interpretation . |
| Approach: | They propose a gradient-guided search over tokens that finds short adversarial triggers that successfully trigger the target prediction. |
| Outcome: | The proposed algorithm finds short trigger sequences that successfully trigger the target prediction. |
Personal Large Language Model Agents: A Case Study on Tailored Travel Planning (2024.emnlp-industry)
Copied to clipboard
Harmanpreet Singh, Nikhil Verma, Yixiao Wang, Manasa Bharadwaj, Homa Fashandi, Kevin Ferreira, Chul Lee
| Challenge: | Large Language Models (LLMs) are becoming more autonomous and capable of handling real-world tasks through their access to tools, various planning strategies, and memory, referred to as LLM agents. |
| Approach: | They introduce a personalized version of TravelPlanner and establish baselines for personal LLM agents by comparing generic and personal plans. |
| Outcome: | The proposed model encapsulates user-related information, preferences, and personal concepts and provides baselines for personal LLM agents. |
GEMMAS: Graph-based Evaluation Metrics for Multi Agent Systems (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existing evaluations focus on the correctness of the final output, overlooking inefficient communication and poor coordination contribute to redundant reasoning and higher computational costs. |
| Approach: | They propose a graph-based evaluation framework that analyzes the internal collaboration process by modeling agent interactions as a directed acyclic graph. |
| Outcome: | The proposed framework shows that outcome-only metrics are insufficient for evaluating multi-agent performance on GSM8K. |
If Only My CGM Could Speak: A Privacy-Preserving Agent for Question Answering over Continuous Glucose Data (2026.findings-acl)
Copied to clipboard
| Challenge: | Current patient platforms only offer static summaries which do not support inquisitive user queries. |
| Approach: | They propose a framework for question answering over personal glucose data that uses large language models to provide a reasoning engine that selects analytical functions. |
| Outcome: | The proposed framework achieves 94% value accuracy on synthetic queries and 88% on ambiguous real-world queries. |
Entity-Based Knowledge Conflicts in Question Answering (2021.emnlp-main)
Copied to clipboard
| Challenge: | Knowledge-dependent tasks typically use two sources of knowledge: parametric, learned at training time, and contextual, given as a passage at inference time. |
| Approach: | They propose a method to mitigate over-reliance on parametric knowledge, which minimizes hallucination, and improves out-of-distribution generalization by 4% - 7%. |
| Outcome: | The proposed method minimizes hallucination and improves generalization to evolving information by 4% - 7%. |
A Benchmark and Dataset for Post-OCR text correction in Sanskrit (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Sanskrit is a classical language with 30 million manuscripts available for digitisation . however, it is considered to be low-resource when it comes to available digital resources. |
| Approach: | They propose to use a post-OCR text correction dataset to correct errors from OCR predictions from 30 different books in the Indian subcontinent. |
| Outcome: | The proposed model outperforms OCR models on graphemic and lexical levels and shows that it is more accurate than previous models. |