Papers by Nikita Bhutani
From Single to Multi: How LLMs Hallucinate in Multi-Document Summarization (2025.findings-naacl)
Copied to clipboard
| Challenge: | a recent study investigated hallucinations in multi-document summarization tasks . but, it is unclear how challenges arising from handling multiple documents affect outputs . |
| Approach: | They investigate how hallucinations manifest in large language models when summarizing topic-specific information from a set of documents. |
| Outcome: | The proposed benchmarks show that the models generate more hallucinations than baselines . the results highlight the need for more effective approaches to mitigate hallucinosity in MDS . |
Retrieval Helps or Hurts? A Deeper Dive into the Efficacy of Retrieval Augmentation to Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LMs) excel in retrieving popular facts, but encounter difficulty with infrequent entity-relation pairs compared to retrievers. |
| Approach: | They propose to use a WiTQA dataset to explore the effects of combinations of entities and relations on LMs. |
| Outcome: | The proposed model can retain popular relations of less common entities while retaining the same popular relations. |
XATU: A Fine-grained Instruction-based Benchmark for Explainable Text Updates (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing text editing benchmark datasets contain coarse-grained instructions and lack explainability, resulting in outputs that deviate from intended changes. |
| Approach: | They propose a benchmark specifically designed for fine-grained instruction-based explainable text editing. |
| Outcome: | The proposed benchmark incorporates fine-grained instructions and gold-standard edit explanations. |
Can Edge Probing Tests Reveal Linguistic Knowledge in QA Models? (2022.coling-1)
Copied to clipboard
| Challenge: | grammatical knowledge is encoded in large pre-trained language models (LMs) this is done through supervised classification tasks to predict the grammamatical properties of a span using only the token representations coming from the LM encoder. |
| Approach: | They propose to use a supervised 'edge probing' task to detect grammatical knowledge in large pre-trained language models (LMs) this is done by encoding grammamatical properties using only token representations coming from the LM encoder. |
| Outcome: | The proposed model performs well when fine-tuned or in adversarial situations where the model is forced to learn wrong correlations. |
Low-resource Entity Set Expansion: A Comprehensive Study on User-generated Text (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing benchmarks for entity set expansion (ESE) are limited to well-formed text and well-defined concepts. |
| Approach: | They propose to use user-generated text to assess the generalizability of ESE methods by identifying phenomena such as non-named entities, multifaceted entities and vague concepts. |
| Outcome: | The proposed methods are based on user-generated text to assess their generalizability and performance. |
Less is More for Long Document Summary Evaluation by LLMs (2024.eacl-short)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown promising performance in summary evaluation tasks, yet they face challenges such as high computational cost and the Lost-in-the-middle problem where important information in the middle of long documents is often overlooked. |
| Approach: | They propose a novel method which extracts key sentences from a long source document and then evaluates the summary by prompting LLMs. |
| Outcome: | The proposed method significantly reduces evaluation costs and exhibits a higher correlation with human evaluations. |
CompactIE: Compact Facts in Open Information Extraction (2022.naacl-main)
Copied to clipboard
| Challenge: | Despite advances in open information extraction, many systems focus on covering more information over compactness of constituents. |
| Approach: | They propose a neural OpenIE system that produces compact extractions with overlapping constituents by using a pipelined approach. |
| Outcome: | The proposed system produces 1.5x-2x more compact extractions than previous systems, with high precision, establishing a new state-of-the-art in OpenIE. |
SubjQA: A Dataset for Subjectivity and Review Comprehension (2020.emnlp-main)
Copied to clipboard
| Challenge: | Subjectivity is the expression of internal opinions or beliefs which cannot be objectively observed or verified. |
| Approach: | They develop a dataset which investigates subjectivity in question answering . they find that subjectivity is an important feature in the case of QA . |
| Outcome: | The proposed dataset shows that subjectivity is an important feature in question answering (QA) it also shows that subjective questions and answers can have more complex interactions than previously thought. |
Efficient Context Selection for Long-Context QA: No Tuning, No Iteration, Just Adaptive‐k (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing adaptive methods struggle with aggregation QA where optimal external context is unknown and variable. |
| Approach: | They propose a single-pass method that selects a query-specific number of passages . Adaptivek retrieval matches or outperforms fixedk baselines while using 10x fewer tokens compared to full-context input . |
| Outcome: | Adaptivek retrieval matches or outperforms fixedk baselines on factoid and aggregation QA benchmarks . it uses 10x fewer tokens than full-context input and still retrieves 70% of relevant passages compared to previous methods . |
Evaluating Bias in LLMs for Job-Resume Matching: Gender, Race, and Education (2025.naacl-industry)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have potential to automate hiring but inherent biases may lead to unfair hiring practices. |
| Approach: | They evaluate how factors such as gender, race, and educational background influence model decisions. |
| Outcome: | The proposed model reduces biases related to gender and race, but implicit biase concerning educational background remains significant. |
Natural Language Processing for Human Resources: A Survey (2025.naacl-industry)
Copied to clipboard
| Challenge: | Recent advances in NLP have the potential to transform HR processes, from recruitment to employee management. |
| Approach: | They analyze key tasks such as information extraction and text classification and their roles in downstream applications like recommendation and language generation while discussing ethical concerns. |
| Outcome: | The proposed frameworks can be applied to HR tasks and to recommendation, language generation, and interaction. |
Exploiting Structure in Representation of Named Entities using Active Learning (C18-1)
Copied to clipboard
| Challenge: | Named entities are atomic objects of reference and reasoning in many knowledge-centric applications. |
| Approach: | They propose an active-learning based framework that drastically reduces the labeled data required to learn entities' structures. |
| Outcome: | The proposed framework outperforms handwritten programs and supervised learning models in relation extraction and entity resolution tasks. |
Open Information Extraction from Question-Answer Pairs (N19-1)
Copied to clipboard
| Challenge: | Existing work on OpenIE extracts structured data from sentences . a system for extracting tuples from question-answer pairs solves this problem . |
| Approach: | They propose a system for extracting tuples from question-answer pairs . they use distributed representations of a question and an answer to generate knowledge facts . |
| Outcome: | The proposed system extracts meaningful structured tuples from question-answer pairs . it can find new and interesting facts to extend knowledge bases, the authors show . |