Papers by Manas Gaur
Towards Robust Evaluation of Unlearning in LLMs via Data Transformations (2024.findings-emnlp)
Copied to clipboard
Abhinav Joshi, Shaswati Saha, Divyaksh Shukla, Sriram Vema, Harsh Jhamtani, Manas Gaur, Ashutosh Modi
| Challenge: | Large Language Models (LLMs) have shown to be a great success in a wide range of applications ranging from regular NLP-based use cases to AI agents. |
| Approach: | They examine the robustness of existing MUL techniques for their ability to enable leakage-proof forgetting in LLMs. |
| Outcome: | The proposed methods can be used to enable leakage-proof forgetting in LLMs. |
IMRNNs: An Efficient Method for Interpretable Dense Retrieval via Embedding Modulation (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing dense retrieval methods rely on static embeddings that obscure bidirectional relationship between queries and documents. |
| Approach: | They propose a framework that augments any black-box dense retrievers with dynamic, bidirectional modulation at inference time. |
| Outcome: | a new framework augments any dense retriever with dynamic, bidirectional modulation at inference time. |
(CPER) From Guessing to Asking: An Approach to Resolving Persona Knowledge Gap in LLMs during Multi-Turn Conversations (2025.naacl-srw)
Copied to clipboard
| Challenge: | Existing methods for identifying and resolving persona knowledge gaps are underexplored. |
| Approach: | They propose a framework that dynamically detects and resolves persona knowledge gaps using intrinsic uncertainty quantification and feedback-driven refinement. |
| Outcome: | The proposed framework detects and resolves persona knowledge gaps using intrinsic uncertainty quantification and feedback-driven refinement on two real-world datasets: CCPE-M for preferential movie recommendations and ESConv for mental health support. |
Experiments or Outcomes? Probing Scientific Feasibility in Large Language Models (2026.acl-short)
Copied to clipboard
| Challenge: | Scientific feasibility assessment asks whether a claim aligns with established knowledge and whether experimental evidence could support or refute it. |
| Approach: | They frame scientific feasibility assessment as a diagnostic reasoning task . given a hypothesis, a model predicts feasible or infeasible and justifies its decision . they evaluate large language models under controlled knowledge conditions . |
| Outcome: | The results show that providing outcome evidence is more reliable than providing experiment descriptions. |
Do LLMs Adhere to Label Definitions? Examining Their Receptivity to External Label Definitions (2025.emnlp-main)
Copied to clipboard
Seyedali Mohammadi, Bhaskara Hanuma Vedula, Hemank Lamba, Edward Raff, Ponnurangam Kumaraguru, Francis Ferraro, Manas Gaur
| Challenge: | Exact label definitions are considered as clues to disambiguate unclear labels, helping models perform their tasks more effectively. |
| Approach: | They conducted controlled experiments on multiple explanation benchmark datasets and label definition conditions using expert-curated, LLM-generated, perturbed, and swapped definitions. |
| Outcome: | The results suggest that models often default to internal representations, particularly in general tasks, while domain-specific tasks benefit more from explicit definitions. |
A Risk-Averse Mechanism for Suicidality Assessment on Social Media (2022.acl-short)
Copied to clipboard
| Challenge: | Social media has become a platform for users to express suicidal thoughts outside traditional clinical settings. |
| Approach: | They propose a risk-averse hierarchical attention classifier that refrains from making uncertain predictions on real-world Reddit data. |
| Outcome: | The proposed system can refrain from 83% of incorrect predictions on real-world Reddit data. |
Beyond Memorization: Testing LLM Reasoning on Unseen Theory of Computation Tasks (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated strong performance on formal language tasks, yet whether this reflects genuine symbolic reasoning or pattern matching on familiar constructions remains unclear. |
| Approach: | They propose a benchmark for deterministic finite automata (DFA) construction from regular languages, comprising factual knowledge questions, seen construction problems from public sources, and unseen problems. |
| Outcome: | The proposed model achieves perfect accuracy on factual questions and 84-90% on seen tasks, but falls sharply on unseen problems (by 30-64%), with failures stemming from systematic misinterpretation of language constraints, incorrect handling of Kleene-star semantics, and a failure to preserve global consistency. |
Side Effects of Erasing Concepts from Diffusion Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Concept erasure techniques (CETs) are used to erade text-to-image (T2I) generative models infringing on privacy, copyright and safety. |
| Approach: | They propose a side effect evaluation benchmark to measure the robustness of CETs. |
| Outcome: | The proposed techniques can be easily circumvented by hierarchy, prompts, and compositional variants of the target. |
SaGE: Evaluating Moral Consistency in Large Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies on Large Language Models (LLMs) have focused on accuracy but lack universally agreed-upon answers for moral scenarios. |
| Approach: | They propose a measure called Semantic Graph Entropy to measure a model's moral consistency grounded in "Rules of Thumb" they construct a moral Consistency Corpus (MCC) with 50K moral questions and the RoTs they followed to investigate LLM consistency on two popular datasets. |
| Outcome: | The proposed measure measures moral consistency on two popular datasets . |