Papers by Santu Rana
ALPACA AGAINST VICUNA: Using LLMs to Uncover Memorization of LLMs (2025.naacl-long)
Copied to clipboard
Aly M. Kassem, Omar Mahmoud, Niloofar Mireshghallah, Hyunwoo Kim, Yulia Tsvetkov, Yejin Choi, Sherif Saad, Santu Rana
| Challenge: | Existing studies have shown that pre-trained LLMs emit training data up to 150 more often than in regular operation. |
| Approach: | They propose a black-box prompt optimization method where an attacker LLM agent uncovers higher levels of memorization in a victim agent . |
| Outcome: | The proposed method shows 23.7% more overlap with training data compared to state-of-the-art baselines. |
The Unintended Trade-off of AI Alignment: Balancing Hallucination Mitigation and Safety in LLMs (2026.findings-eacl)
Copied to clipboard
| Challenge: | Hallucination in large language models has been studied, but a side effect remains unrecognized . a new study examines the trade-off between truthfulness and safety alignment . |
| Approach: | They propose a method that disentangles hallucination from hallucinian features using sparse autoencoders. |
| Outcome: | The proposed method preserves refusal behavior and task utility while maintaining safety alignment. |