Papers by Anshu Bhatia
Masked Audio Text Encoders are Effective Multi-Modal Rescorers (2023.findings-acl)
Copied to clipboard
| Challenge: | Masked Language Models (MLMs) have proven to be effective for second-pass rescoring in Automatic Speech Recognition systems. |
| Approach: | They propose a multi-modal masked language model rescorer which integrates acoustic representations into the input space of MLM. |
| Outcome: | The proposed model reduces word error rate (WER) by 4%-16% on in-domain and 3%-7% on out-of-domain datasets over the text-only baseline. |
SpeechGuard: Exploring the Adversarial Robustness of Multi-modal Large Language Models (2024.findings-acl)
Copied to clipboard
Raghuveer Peri, Sai Muralidhar Jayanthi, Srikanth Ronanki, Anshu Bhatia, Karel Mundnich, Saket Dingliwal, Nilaksh Das, Zejiang Hou, Goeric Huybrechts, Srikanth Vishnubhotla, Daniel Garcia-Romero, Sundararajan Srinivasan, Kyu Han, Katrin Kirchhoff
| Challenge: | Integrated Speech and Large Language Models (SLMs) that follow speech instructions and generate relevant text responses have gained popularity lately. |
| Approach: | They propose algorithms that can generate adversarial examples to jailbreak SLMs without human involvement. |
| Outcome: | The proposed algorithms achieve state-of-the-art on spoken question-answering task scoring over 80% on both safety and helpfulness metrics. |