Papers by Murali Annavaram
Ethos: Rectifying Language Models in Orthogonal Parameter Space (2024.findings-naacl)
Copied to clipboard
| Challenge: | Language models (LMs) generate toxic, biased content and reveal private training records. |
| Approach: | They propose an efficient approach that rectifies LMs to mitigate toxicity and bias . Ethos distinguishes general beneficial and undesired knowledge when reconstructing task vectors . |
| Outcome: | The proposed approach mitigates toxicity and bias in outputs and avoids privacy leakage. |
Estimating Privacy Leakage of Augmented Contextual Knowledge in Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Prior work has shown that privacy leakage of parametric knowledge often occurs from memorized pre-training data. |
| Approach: | They propose a metric that builds on differential privacy to estimate the privacy leakage of contextual knowledge during decoding by comparing parametric and contextual knowledge. |
| Outcome: | The proposed method overestimates the privacy leakage of parametric knowledge while separating parametric and contextual knowledge. |
Differentially Private Next-Token Prediction of Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are becoming increasingly important for ensuring privacy, but DP-SGD overestimates an adversary’s capabilities in having white box access. |
| Approach: | They propose a private prediction protocol for next-token prediction that utilizes the inherent stochasticity of next-toning and a public model to achieve Differential Privacy. |
| Outcome: | The proposed protocol outperforms DP-SGD and DP training methods for privacy on large datasets. |
On Using Arabic Language Dialects in Recommendation Systems (2025.findings-naacl)
Copied to clipboard
| Challenge: | Using natural language processing (NLP) to analyze user reviews in recommendation systems is unexplored. |
| Approach: | They propose to integrate Arabic dialects as a signal in recommendation systems by using explicit and implicit approaches. |
| Outcome: | The proposed approach improves recommendation performance and encourages further research in the Arab multicultural world. |
KVPR: Efficient LLM Inference with I/O-Aware KV Cache Partial Recomputation (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to inference large language models are limited by CPU capabilities and memory constraints. |
| Approach: | They propose an efficient I/O-aware LLM inference method that overlaps GPU computation with KV cache transfer to minimize idle GPU time. |
| Outcome: | The proposed method reduces the cost of auto-regressive decoding by 35.8% . it also achieves 46.2% higher throughput during decoding compared to state-of-the-art methods. |
Differentially Private Knowledge Distillation via Synthetic Text Generation (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language models (LLMs) are achieving state-of-the-art performance in many downstream tasks, but data privacy is a major challenge for practitioners. |
| Approach: | They propose a differentially private knowledge distillation algorithm that exploits the knowledge of a teacher LLM and a student's output distribution. |
| Outcome: | The proposed algorithm significantly improves the utility over baselines on the Big Patent dataset, with strong privacy parameters, =2. |
EQUIP: EQUivariant preserving In-Place updates for Efficient Token Pruning (2026.acl-long)
Copied to clipboard
| Challenge: | Token-pruning methods cause "holes" in KV tensors, posing major challenges . equip reduces recomputation of rotation operations through in-place update, caching and re-indexing . |
| Approach: | They propose an EQUIP-based in-place token update mechanism that preserves the equivariance property of the operations performed in the attention computation. |
| Outcome: | EQUIP reduces recomputation of rotation operations and reduces eviction overheads . it achieves geomean speedups of 1.62 (or 1.47) over StreamingLLM and 3.45 ( or 1.86) |
StATIK: Structure and Text for Inductive Knowledge Graph Completion (2022.findings-naacl)
Copied to clipboard
Elan Markowitz, Keshav Balasubramanian, Mehrnoosh Mirtaheri, Murali Annavaram, Aram Galstyan, Greg Ver Steeg
| Challenge: | Knowledge graphs (KGs) represent incomplete knowledge bases. |
| Approach: | They propose to use language models to extract semantic information from text descriptions while using Message Passing Neural Networks to capture structural information. |
| Outcome: | The proposed model achieves state of the art on three challenging inductive baselines. |
MobiZO: Enabling Efficient LLM Fine-Tuning at the Edge via Inference Engines (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are currently pre-trained and fine-tuned on large cloud servers . fine-timing on resource-constrained edge devices presents significant memory and computational demands . |
| Approach: | They propose a resource-efficient fine-tuning framework for LLMs specifically designed for edge devices. |
| Outcome: | Experiments show that MobiZO achieves substantial runtime speedups and memory savings while improving fine-tuning accuracy. |
DELTA: Dynamic Layer-Aware Token Attention for Efficient Long-Context Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Large reasoning models generate long chains of intermediate steps, but their inference cost is dominated by decoding, where each new token must attend to the entire growing sequence. |
| Approach: | They propose a training-free sparse attention mechanism that reduces inference cost by evicting entries from the key-value cache. |
| Outcome: | The proposed model matches or surpasses full attention on reasoning benchmarks . it reduces the number of attended tokens by up to 4.25 and delivers 1.54 speedup . |
Mind the Dialect: NLP Advancements Uncover Fairness Disparities for Arabic Users in Recommendation Systems (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that recommendation systems can exhibit unfair behavior when performance varies across users . the authors highlight the intersection of NLP and recommendation system research . |
| Approach: | They investigate fairness disparities in recommendation quality among Arabic-speaking users . arab-speaking people's dialectal diversity is underrepresented in recommendation system research . |
| Outcome: | The authors highlight the intersection of NLP and recommendation systems . their findings highlight the broader social impact of N. |