Papers by Zhan Qin
DELMAN: Dynamic Defense Against Large Language Model Jailbreaking with Model Editing (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing safety mechanisms for Large Language Models (LLMs) are inadequate to protect against jailbreak attacks, resulting in performance degradation on general tasks. |
| Approach: | They propose a method that directly updates a minimal set of relevant parameters to neutralize harmful behaviors while preserving the model’s utility. |
| Outcome: | The proposed model outperforms baseline methods in mitigating jailbreak attacks while preserving the model’s utility. |
Don’t Say No: Jailbreaking LLM by Suppressing Refusal (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are vulnerable to "jailbreaking" attacks where crafted prompts manipulate them into producing toxic content. |
| Approach: | They propose to improve the target loss objective by combining a cosine decay schedule method with refusal suppression to achieve higher success rates. |
| Outcome: | The proposed approach outperforms baseline attacks and achieves state-of-the-art attack success rates. |
Cross-modality Information Check for Detecting Jailbreaking in Multimodal Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Multimodal Large Language Models (MLLMs) are susceptible to jailbreak attacks, authors say . multimodal information increases the risk of attacks, but also provides additional data . |
| Approach: | They propose a jailbreaking detector that detects maliciously perturbed image inputs . cross-modality information detector is designed to detect cross-modal similarity between harmful queries and adversarial images. |
| Outcome: | a new tool can detect maliciously perturbed image inputs without modification or computation cost. |
Synchronized Video Storytelling: Generating Video Narrations with Structured Storyline (2024.acl-long)
Copied to clipboard
| Challenge: | Existing studies on dense video captioning and video story generation have made some progress, but in practical applications, we typically require synchronized narrations for ongoing visual scenes. |
| Approach: | They propose a task of Synchronized Video Storytelling to generate synchronized narrations for videos using a benchmark dataset with rich annotations. |
| Outcome: | The proposed framework can generate narrations with the guidance of the generated or predefined storyline and human evaluations validate the effectiveness. |
Premise-based Multimodal Reasoning: Conditional Inference on Joint Textual and Visual Clues (2022.acl-long)
Copied to clipboard
Qingxiu Dong, Ziwei Qin, Heming Xia, Tian Feng, Shoujie Tong, Haoran Meng, Lin Xu, Zhongyu Wei, Weidong Zhan, Baobao Chang, Sujian Li, Tianyu Liu, Zhifang Sui
| Challenge: | Existing work in vision language cross-modal reasoning uses binary or multi-choice classification based on source image and textual query. |
| Approach: | They propose a task where a textual premise is the background presumption on each source image. |
| Outcome: | The proposed task is based on a dataset of 15,360 movie screenshots and human-curated premise templates from 6 pre-defined categories. |
Towards Identification and Intervention of Safety-Critical Parameters in Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing safety-related methodologies for large language models are lacking . despite advances in safety alignment techniques, safeguarding LLMs during adaptation to various tasks remains a challenge. |
| Approach: | They propose a framework to quantify how different parameters affect LLM safety . they propose two targeted intervention paradigms for safety enhancement and preservation . |
| Outcome: | The proposed framework reveals safety-critical patterns across different LLM architectures. |
Uncertainty-Aware Contrastive Sentence Embedding With Local Context Representation for Text Classification (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing models for text classification are based on encoder-only transformers and generative pre-trained transformers. |
| Approach: | They propose an uncertainty-aware contrastive sentence embedding approach that addresses language ambiguity and inter-class separability for a text classification task. |
| Outcome: | The proposed approach improves classification accuracy on public datasets compared with state-of-the-art methods. |