Papers by Minghong Fang
GradSafe: Detecting Jailbreak Prompts for LLMs via Safety-Critical Gradient Analysis (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for detecting jailbreak prompts are primarily online moderation APIs or finetuned LLMs. |
| Approach: | They propose a method which scrutinizes the gradients of safety-critical parameters in large LLMs to detect jailbreak prompts. |
| Outcome: | The proposed method outperforms Llama Guard in detecting jailbreak prompts despite extensive finetuning with a large dataset. |