Papers by Anil Ramakrishna
LUME: LLM Unlearning with Multitask Evaluations (2025.findings-emnlp)
Copied to clipboard
Anil Ramakrishna, Yixin Wan, Xiaomeng Jin, Kai-Wei Chang, Zhiqi Bu, Bhanukiran Vinzamuri, Volkan Cevher, Mingyi Hong, Rahul Gupta
| Challenge: | Unlearning aims to remove copyrighted, sensitive, or private content from large language models without a full retraining. |
| Approach: | They propose a multi-task unlearning benchmark LUME that unlearns short novels, biographies and public biographie . |
| Outcome: | The proposed benchmark unlearns short novels, biographies and public biographie . it also releases fine-tuned models with 1B and 7B parameter sizes as targets . |
Do Not Design, Learn: A Trainable Scoring Function for Uncertainty Estimation in Generative LLMs (2025.findings-naacl)
Copied to clipboard
Duygu Nur Yaldiz, Yavuz Faruk Bakman, Baturalp Buyukates, Chenyang Tao, Anil Ramakrishna, Dimitrios Dimitriadis, Jieyu Zhao, Salman Avestimehr
| Challenge: | Existing methods for probability-based UE are limited by their inability to handle biased probabilities and complex semantic dependencies between tokens. |
| Approach: | They propose a learning-based scoring function that captures complex dependencies between tokens and probabilities and produces more reliable responses. |
| Outcome: | The proposed function outperforms existing scoring functions in question-answering and arithmetical reasoning tasks with different datasets. |
Explaining and Improving Contrastive Decoding by Extrapolating the Probabilities of a Huge and Hypothetical LM (2024.emnlp-main)
Copied to clipboard
| Challenge: | Contrastive decoding (CD) improves the next-token distribution of a large expert language model (LM) using a small amateur LM. |
| Approach: | They propose a new unsupervised decoding method called Asymptotic Probability Decoding (APD) that extrapolates the probability curves from the LMs of different sizes to infer the asymptototic probabilities from an infinitely large LM. |
| Outcome: | The proposed method improves the next-token distribution of a large expert language model using a small amateur LM. |
From Narrow Unlearning to Emergent Misalignment in LLMs (2026.acl-short)
Copied to clipboard
Erum Mushtaq, Anil Ramakrishna, Satyapriya Krishna, Sattvik Sahai, Prasoon Goyal, Kai-Wei Chang, Tao Zhang, Rahul Gupta
| Challenge: | Recent work shows that fine-tuning on insecure code data can trigger an emergent misalignment (EMA) phenomenon . |
| Approach: | They extend their study by demonstrating that EMA can arise from narrow refusal unlearning . they perform refusal unLearning on Cybersecurity and Safety concept and evaluate EMA . |
| Outcome: | The proposed model can generate malicious responses even to unrelated prompts . the proposed model is able to restore alignment across the affected domains while having lower refusal rates. |
On Localizing and Deleting Toxic Memories in Large Language Models (2025.findings-naacl)
Copied to clipboard
Anubrata Das, Manoj Kumar, Ninareh Mehrabi, Anil Ramakrishna, Anna Rumshisky, Kai-Wei Chang, Aram Galstyan, Morteza Ziyadi, Rahul Gupta
| Challenge: | Existing methods to reduce toxic generation in large language models are not fully understood. |
| Approach: | They propose to understand the mechanisms that drive toxic generation in large language models by using memory localization to reduce toxic generation. |
| Outcome: | The proposed method reduces toxic generation from 62.86% to 28.61%, but it also improves generation quality. |
Unlearning as multi-task optimization: A normalized gradient difference approach with an adaptive learning rate (2025.naacl-long)
Copied to clipboard
Xiaomeng Jin, Zhiqi Bu, Bhanukiran Vinzamuri, Anil Ramakrishna, Kai-Wei Chang, Volkan Cevher, Mingyi Hong
| Challenge: | Existing methods to remove unwanted knowledge from large language models are formulated as minimizing memorization through the loss of the model. |
| Approach: | They propose a normalized gradient difference algorithm that optimizes a forgetting objective and an automatic learning rate scheduler that allows for better control over the trade-off between the objectives. |
| Outcome: | The proposed method improves on TOFU and MUSE datasets while exhibiting stable training. |
INVITE: a Testbed of Automatically Generated Invalid Questions to Evaluate Large Language Models for Hallucinations (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in Large language models have enabled them to hold free form conversations over multiple turns, but they exhibit a tendency to make unfounded and incorrect statements, commonly labeled as hallucinations. |
| Approach: | They propose a framework to test large language models for hallucinations using automatically generated INValId questions. |
| Outcome: | The proposed framework is based on a testbed of automatically generated INValId questions to evaluate large language models for hallucinations. |
Attribute Controlled Fine-tuning for Large Language Models: A Case Study on Detoxification (2024.findings-emnlp)
Copied to clipboard
Tao Meng, Ninareh Mehrabi, Palash Goyal, Anil Ramakrishna, Aram Galstyan, Richard Zemel, Kai-Wei Chang, Rahul Gupta, Charith Peris
| Challenge: | Using a sequence-level constraint, we regularize the LLMtraining by penalizing the KL divergence between the desired output distribution and the LRM’s posterior. |
| Approach: | They propose a constraint learning schema forfine-tuning Large Language Models with attribute control by penalizing the KL divergence be-tween the desired output distribution and the LLM's posterior. |
| Outcome: | The proposed approach improves the performance of large language models while enhancing their utility and generation quality. |
Not Every Token Needs Forgetting: Selective Unlearning Balancing Forgetting and Utility in Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Conventional unlearning approaches forget all tokens in a target document, including common tokens that carry general knowledge. |
| Approach: | They propose a method that identifies a critical subset of tokens within the forgetting set that is relevant to the unwanted information and unlearns only those tokens. |
| Outcome: | Experiments on two benchmarks and six baseline unlearning algorithms show that selective unlearning achieves effective unlearning on the targeted forget data. |
Towards Safety Reasoning in LLMs: AI-agentic Deliberation for Policy-embedded CoT Data Creation (2025.findings-acl)
Copied to clipboard
Tharindu Kumarage, Ninareh Mehrabi, Anil Ramakrishna, Xinyan Zhao, Richard Zemel, Kai-Wei Chang, Aram Galstyan, Rahul Gupta, Charith Peris
| Challenge: | Safety reasoning paradigms require high-quality policy-embedded chain-of-thought datasets . generating such data through human annotations is prohibitively expensive . |
| Approach: | They propose AIDSAFE: Agentic Iterative Deliberation for Safety Reasoning . AIDS AFE leverages multi-agent deliberation to iteratively expand reasoning on safety policies . |
| Outcome: | The proposed model improves policy adherence and reasoning quality while maintaining acceptable utility and over-refusal accuracy. |
BLUR: A Bi-Level Optimization Approach for LLM Unlearning (2026.eacl-long)
Copied to clipboard
Hadi Reisizadeh, Jinghan Jia, Zhiqi Bu, Bhanukiran Vinzamuri, Anil Ramakrishna, Kai-Wei Chang, Volkan Cevher, Sijia Liu, Mingyi Hong
| Challenge: | Existing algorithms to unlearn knowledge and capabilities from large datasets are unclear how to best formulate the unlearning problem. |
| Approach: | They propose to model the hierarchical structure of the unlearning problem, where the forget problem takes priority over the retain problem, and propose an algorithm that aims to unlearn knowledge and capabilities. |
| Outcome: | The proposed algorithm outperforms all state-of-the-art algorithms across unlearning tasks, models, and metrics. |
Towards Realistic Single-Task Continuous Learning Research for NER (2021.findings-emnlp)
Copied to clipboard
Justin Payan, Yuval Merhav, He Xie, Satyapriya Krishna, Anil Ramakrishna, Mukund Sridhar, Rahul Gupta
| Challenge: | Academic datasets are often static and contain data that is annotated all at once based on fixed annotation guidelines. |
| Approach: | They propose to build a single-task continuous learning dataset from an existing dataset and release it along with the code to the research community. |
| Outcome: | The proposed model is based on an existing dataset and released to the research community. |
Federated Learning with Noisy User Feedback (2022.naacl-main)
Copied to clipboard
Rahul Sharma, Anil Ramakrishna, Ansel MacLaughlin, Anna Rumshisky, Jimit Majmudar, Clement Chung, Salman Avestimehr, Rahul Gupta
| Challenge: | Artificial Intelligence (AI) and Machine Learning (ML) systems are becoming more popular and are causing concerns over user privacy. |
| Approach: | They propose a method for training ML models using positive and negative user feedback and a framework to extract labels on edge to make FL viable. |
| Outcome: | The proposed method improves significantly over a self-training baseline, achieving performance closer to models trained with full supervision. |
Improving Large-Scale Conversational Assistants using Model Interpretation based Training Sample Selection (2022.emnlp-industry)
Copied to clipboard
Stefan Schroedl, Manoj Kumar, Kiana Hajebi, Morteza Ziyadi, Sriram Venkatapathy, Anil Ramakrishna, Rahul Gupta, Pradeep Natarajan
| Challenge: | Large-scale, voice-based conversational assistants process each utterance through a multi-stage pipeline that includes wakeword detection, automatic speech recognition (ASR), natural language understanding (NLU), entity resolution, and textto-speech. |
| Approach: | They propose a method to identify customer implicitly satisfied with Alexa's responses by leveraging interpretations of model behavior. |
| Outcome: | The proposed approach produces statistically significant improvements in both offline and online tests. |
SWAN: Semantic Watermarking with Abstract Meaning Representation (2026.acl-long)
Copied to clipboard
Ziping Ye, Gourab Dey, Christos Christodoulopoulos, Charith Peris, Anil Ramakrishna, Weitong Ruan, Aram Galstyan, Kai-Wei Chang, Rahul Gupta, Ninareh Mehrabi
| Challenge: | Existing methods to embed signatures by adjusting token selection preferences during text generation are highly sensitive to paraphrasing and synonyms. |
| Approach: | They propose a framework that embeds watermark signatures into the semantic structure of a sentence using Abstract Meaning Representation (AMR). |
| Outcome: | Empirical evaluation shows SWAN matches state-of-the-art detection performance on unaltered watermarked text while improving robustness against paraphrasing. |
Tree-of-Traversals: A Zero-Shot Reasoning Algorithm for Augmenting Black-box Language Models with Knowledge Graphs (2024.acl-long)
Copied to clipboard
Elan Markowitz, Anil Ramakrishna, Jwala Dhamala, Ninareh Mehrabi, Charith Peris, Rahul Gupta, Kai-Wei Chang, Aram Galstyan
| Challenge: | Knowledge graphs (KGs) complement Large Language Models (LLMs) by providing reliable, structured, domain-specific, and up-to-date external knowledge. |
| Approach: | They propose a zero-shot reasoning algorithm that augments black-box LLMs with one or more KGs. |
| Outcome: | The proposed algorithm significantly improves performance on question answering and KG question answering tasks. |
Correcting Language Model Outputs by Editing Salient Layers (2024.findings-eacl)
Copied to clipboard
| Challenge: | Existing models that use manual layer selection require prior domain knowledge and expensive empirical layer selection methods. |
| Approach: | They propose a model editing approach that selectively edits a small subset of model parameters to update the factual knowledge. |
| Outcome: | The proposed solution matches the accuracy of previous approaches with only 1/3 of their edits, enabling efficient updates to the parametric knowledge in large language models. |