Papers by Bhavya Bhavya
SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearning (2024.emnlp-main)
Copied to clipboard
Jinghan Jia, Yihua Zhang, Yimeng Zhang, Jiancheng Liu, Bharat Runwal, James Diffenderfer, Bhavya Kailkhura, Sijia Liu
| Challenge: | Large Language Models (LLMs) have highlighted the need for effective unlearning mechanisms to comply with data regulations and ethical AI practices. |
| Approach: | They propose a second-order optimization-based LLM unlearning framework which extends the static, one-shot model update using influence unlearning to a dynamic, iterative unlearning process. |
| Outcome: | The proposed framework outperforms first-order methods across unlearning tasks, models, and metrics. |
Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Models (2024.acl-long)
Copied to clipboard
Jinhao Duan, Hao Cheng, Shiqi Wang, Alex Zavalny, Chenan Wang, Renjing Xu, Bhavya Kailkhura, Kaidi Xu
| Challenge: | Large Language Models (LLMs) show promising results in language generation but often “hallucinate”, making their outputs less reliable. |
| Approach: | They propose to shift attention to more relevant components at token- and sentence-levels for better UQ. |
| Outcome: | The proposed approach improves the performance of a range of popular “off-the-shelf” LLMs with model sizes extending up to 33B parameters. |
Speculative Diffusion Decoding: Accelerating Language Generation through Diffusion (2025.naacl-long)
Copied to clipboard
Jacob K Christopher, Brian R. Bartoldson, Tal Ben-Nun, Michael Cardei, Bhavya Kailkhura, Ferdinando Fioretto
| Challenge: | Existing methods to accelerate large language model inference are limited by the reliance on incremental token generation in existing draft models. |
| Approach: | They propose an adaptation of speculative decoding which uses discrete diffusion models to generate draft sequences and allows parallelization of both the drafting and verification steps. |
| Outcome: | The proposed approach provides 7.2x speedups over standard generation processes and 1.75x speed ups over existing speculative decoding approaches. |
AnaDE1.0: A Novel Data Set for Benchmarking Analogy Detection and Extraction (2024.eacl-long)
Copied to clipboard
| Challenge: | Textual analogies that make comparisons between two concepts are often used for explaining complex ideas, creative writing, and scientific discovery. |
| Approach: | They propose a task that includes three synergistic tasks: detecting documents containing analogies, extracting text segments that make up the analogy, and identifying the (source and target) concepts being compared. |
| Outcome: | The proposed task performs well on all sub-tasks and smaller models perform better than non-finetuned ChatGPT, suggesting high task difficulty. |
Layer-Level Self-Exposure and Patch: Affirmative Token Mitigation for Jailbreak Attack Defense (2025.naacl-long)
Copied to clipboard
Yang Ouyang, Hengrui Gu, Shuhang Lin, Wenyue Hua, Jie Peng, Bhavya Kailkhura, Meijun Gao, Tianlong Chen, Kaixiong Zhou
| Challenge: | Existing methods to defend against jailbreak attacks exploit vulnerabilities to elicit unintended or harmful outputs. |
| Approach: | They propose a method to defend against jailbreak attacks by patching specific layers within large language models through self-augmented datasets. |
| Outcome: | The proposed approach reduces harmfulness and attack success rate of jailbreak attacks without compromising utility for benign queries compared to previous methods. |
GRNFormer: A Biologically-Guided Framework for Integrating Gene Regulatory Networks into RNA Foundation Models (2025.findings-acl)
Copied to clipboard
Mufan Qiu, Xinyu Hu, Fengwei Zhan, Sukwon Yun, Jie Peng, Ruichen Zhang, Bhavya Kailkhura, Jiekun Yang, Tianlong Chen
| Challenge: | Foundation models for single-cell RNA sequencing ignore biological prior knowledge encoded in gene regulatory relationships and fail to leverage multi-omics signals. |
| Approach: | They propose a framework that integrates multi-scale gene regulatory networks into RNA foundation model training. |
| Outcome: | The proposed framework improves on state-of-the-art models on three downstream tasks . it integrates multi-scale gene regulatory networks (GRNs) from multi-omics data into training . |
Extracting and Understanding the Superficial Knowledge in Alignment (2025.naacl-long)
Copied to clipboard
Runjin Chen, Gabriel Jacob Perin, Xuxi Chen, Xilun Chen, Yan Han, Nina S. T. Hirata, Junyuan Hong, Bhavya Kailkhura
| Challenge: | Recent studies have shown that alignment of large language models with human values and preferences requires substantial data and computation resources. |
| Approach: | They propose a method to extract and isolate superficial knowledge from aligned models by focusing on the shallow modifications to the final token selection process. |
| Outcome: | The proposed method extracts and isolates superficial knowledge from aligned models, focusing on the shallow modifications to the final token selection process. |
RankMean: Module-Level Importance Score for Merging Fine-tuned LLM Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Traditionally, developing new language models involves fine-tuning pre-trained LMs . model merging is a cost-effective alternative to fine-timing LM models for multiple tasks . |
| Approach: | They propose an algorithm for merging fine-tuned language models without additional training. |
| Outcome: | The proposed algorithm outperforms baseline methods on multiple benchmarks. |
ReTA: Recursively Thinking Ahead to Improve the Strategic Reasoning of Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing logical reasoning evaluations of Large Language Models (LLMs) focus on single-turn and static environments, such as arithmetic problems. |
| Approach: | They propose a Recursively Thinking-Ahead agent that analyzes the opponents’ future moves/actions and assigns reward signals for these situations. |
| Outcome: | The proposed agent is based on two scenarios: Online Racing and Offline Probing. |
BenchMarker: An Education-Inspired Toolkit for Highlighting Flaws in Multiple-Choice Benchmarks (2026.acl-long)
Copied to clipboard
Nishant Balepur, Bhavya Rajasekaran, Hyunjin Jane Oh, Michael Xie, Atrey Desai, Vipul Gupta, Steven James Moore, Eunsol Choi, Rachel Rudinger, Jordan Lee Boyd-Graber
| Challenge: | Multiple-choice question answering (MCQ) is standard in NLP, but benchmarks lack rigorous quality control. |
| Approach: | They propose an education-inspired toolkit that uses LLM judges to flag flaws in MCQs . they validate the tool with annotations and run it to audit 12 benchmarks based on 19-rule education rubric . |
| Outcome: | The proposed toolkit flags three common MCQ flaws based on a 19-rule education rubric . contaminated MCqs tend to inflate accuracy, while writing errors lower it and change rankings . |