Papers by Neil Gong
GradSafe: Detecting Jailbreak Prompts for LLMs via Safety-Critical Gradient Analysis (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for detecting jailbreak prompts are primarily online moderation APIs or finetuned LLMs. |
| Approach: | They propose a method which scrutinizes the gradients of safety-critical parameters in large LLMs to detect jailbreak prompts. |
| Outcome: | The proposed method outperforms Llama Guard in detecting jailbreak prompts despite extensive finetuning with a large dataset. |
ReCaLL: Membership Inference via Relative Conditional Log-Likelihoods (2024.emnlp-main)
Copied to clipboard
| Challenge: | ReCaLL (Relative Conditional Log-Likelihood) is a membership inference attack that can detect LLMs’ pretraining data by leveraging their conditional language modeling capabilities. |
| Approach: | They propose a membership inference attack to detect LLMs’ pretraining data by leveraging their conditional language modeling capabilities. |
| Outcome: | The proposed model achieves state-of-the-art performance on the WikiMIA dataset, even with random and synthetic prefixes, and can be further improved using an ensemble approach. |
Visual Hallucinations of Multi-modal Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing studies find VH instances only in existing image datasets, which results in biased understanding of MLLMs’ performance under VH. |
| Approach: | They propose a tool called VHTest to generate a diverse set of VH instances from existing image datasets and a text-to-image generative model to generate VH images based on the text descriptions. |
| Outcome: | The proposed tool finds VH instances in existing image datasets and generates images based on the text descriptions. |