Papers by Vyas Raina
LLM Task Interference: An Initial Study on the Impact of Task-Switch in Conversational History (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in Natural Language Processing (NLP) have led to the widespread deployment of large language models (LLMs) across various applications. |
| Approach: | They propose to formalize the study of task-switches in conversational LLMs by analyzing conversational history. |
| Outcome: | The proposed study formalizes and investigates the sensitivity of large language models to taskswitch scenarios in conversational LLMs. |
Extreme Miscalibration and the Illusion of Adversarial Robustness (2024.acl-long)
Copied to clipboard
| Challenge: | emergence of the Adversarial Training paradigm has shown some success in training models to be more robust to these small adversarial perturbations. |
| Approach: | They propose to use adversarial examples to detect adversarials by miscalibrating models to mask gradients in a way that interferes with adversarial attack search methods. |
| Outcome: | The proposed model gains are an illusion of robustness (IOR) and urge the NLP community to incorporate test-time temperature scaling into their robustness evaluations to ensure that any observed gains are genuine. |
Residue-Based Natural Language Adversarial Attack Detection (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to detect adversarial examples for deep learning based systems focus on image embedding feature spaces . however, existing approaches focus on text features, without considering model embeddable spaces. |
| Approach: | They propose a sentence-embedding “residue” detector to identify adversarial examples from embedded feature spaces. |
| Outcome: | The proposed detector outperforms existing model-focused detectors on many tasks. |
Grammatical Error Correction Systems for Automated Assessment: Are They Susceptible to Universal Adversarial Attacks? (2022.aacl-main)
Copied to clipboard
| Challenge: | With advances in deep learning, GEC systems are susceptible to adversarial attacks, in which a small change at the input can cause large undesired changes at the output. |
| Approach: | They propose to use a concatenative universal attack to deceive the system into not correcting grammatical errors to create the perception of higher language ability. |
| Outcome: | The proposed attack can deceive the system into not correcting (concealing) grammatical errors to create the perception of higher language ability. |
Analyzing Biases to Spurious Correlations in Text Classification Tasks (2022.aacl-short)
Copied to clipboard
| Challenge: | Often these systems exceed human performance, but there is a caveat: standard benchmarks often assume that training and evaluation data are drawn independently and identically from the same underlying distribution. |
| Approach: | They propose to exploit spurious correlations in training data to exploit these correlations . they show that even when only ‘stop’ words are available, it is possible to predict the class significantly better than random. |
| Outcome: | The proposed model can predict class significantly better when only ‘stop’ words are available at the input stage, but can degrade the ability of the system to generalize well to out-of-domain data. |
Funny or Persuasive, but Not Both: Evaluating Fine-Grained Multi-Concept Control in LLMs (2026.eacl-short)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) provide strong generative capabilities, but many applications require explicit and fine-grained control over specific textual concepts. |
| Approach: | They propose a framework for fine-grained controllability for single- and dual-concept scenarios . they find performance drops in the dual-constituency setting, even though chosen concepts should be separable . |
| Outcome: | The proposed framework shows that models struggle with compositionality even when concepts are intuitively independent. |
Muting Whisper: A Universal Acoustic Adversarial Attack on Speech Foundation Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | 'special' tokens in large speech foundation models such as Whisper are used to guide their language generation process, but can be exploited by adversarial attacks to manipulate the model's behavior. |
| Approach: | They propose a method to learn a universal acoustic realization of Whisper's |endoftext|> token, which encourages the model to ignore the speech and only transcribe the special token, effectively muting the model. |
| Outcome: | The proposed method can mute Whisper models for over 97% of speech samples and can be used to bypass speech moderation systems and protect private speech data. |
Is LLM-as-a-Judge Robust? Investigating Universal Adversarial Attacks on Zero-shot LLM Assessment (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are powerful zero-shot assessors used in real-world situations . however, no study has examined the vulnerability of judge-LLM to adversarial manipulation . |
| Approach: | They propose a simple surrogate attack where a surrogated model is attacked and the learned attack phrase transferred to unknown judge-LLMs. |
| Outcome: | The proposed algorithm shows that judge-LLMs can be significantly more susceptible to adversarial attacks when used for absolute scoring, rather than comparative assessment. |
Universal Acoustic Adversarial Attacks for Flexible Control of Speech-LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | acoustic adversarial attacks on speech LLMs may make them more vulnerable to adversarials . flexible speech encoders and large language models have enabled speech Llms to handle a wide range of processing tasks. |
| Approach: | They investigate universal adversarial attacks on speech LLMs by pre-trained speech encoders and large language models. |
| Outcome: | The proposed model can handle a wide range of spoken language processing tasks. |