Papers by Sahil Verma
MULTIGUARD: An Efficient Approach for AI Safety Moderation Across Languages and Modalities (2025.emnlp-main)
Copied to clipboard
Sahil Verma, Keegan Hines, Jeff Bilmes, Charlotte Siska, Luke Zettlemoyer, Hila Gonen, Chandan Singh
| Challenge: | Existing approaches to detect harmful queries to large language models are fallible and vulnerable to attacks that exploit mismatched generalization of model capabilities. |
| Approach: | They propose an approach to detect harmful queries to large language models (LLMs) OMNIGUARD identifies internal representations of an LLM/MLLM that are aligned across languages or modalities and builds a language-agnostic or modality-adic classifier for detecting harmful prompts. |
| Outcome: | OMNIGUARD improves harmful prompt classification accuracy by 11.57% over the strongest baseline in a multilingual setting, by 20.44% for image-based prompts, and sets a new SOTA for audio-based ones. |
AXCEL: Automated eXplainable Consistency Evaluation using LLMs (2024.findings-emnlp)
Copied to clipboard
P Aditya Sreekar, Sahil Verma, Suransh Chopra, Abhishek Persad, Sarik Ghazarian, Narayanan Sadagopan
| Challenge: | Large Language Models (LLMs) are widely used for various tasks but evaluating the consistency of generated text remains a challenge. |
| Approach: | They propose a prompt-based consistency metric which provides explanations for consistency scores by providing detailed reasoning and pinpointing inconsistent text spans. |
| Outcome: | The proposed metric outperforms state-of-the-art metrics in summarization, free text generation and data-to-text conversion tasks by 8.7% and 6.2%. |