Papers by Tianrong Zhang
WordGame: Efficient & Effective LLM Jailbreak via Simultaneous Obfuscation in Query and Response (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent advances in large language models have raised concerns about their susceptibility to jailbreaking attacks, which leads to harmful content inadvertently. |
| Approach: | They propose to exploit the safety alignment patterns of LLMs by simultaneous obfuscation in queries and responses to break down adversarial intent of query. |
| Outcome: | The proposed attack breaks down adversarial intent of query and encourages benign content regarding the games to precede anticipated harmful content in the response. |
PromptFix: Few-shot Backdoor Removal via Adversarial Prompt Tuning (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing studies have shown that pre-trained language models can be backdoored such that model behavior is manipulated when trigger tokens are presented. |
| Approach: | They propose a backdoor mitigation strategy for NLP models via adversarial prompt-tuning in few-shot settings that uses two extra sets of soft tokens which approximate the trigger and counteract it respectively. |
| Outcome: | The proposed method keeps model parameters intact and only utilizes two extra sets of soft tokens which approximate the trigger and counteract it respectively. |