| Challenge: | Pluralism alignment is the goal of creating AI that can coexist with and serve morally multifaceted humanity. |
| Approach: | They propose to use jailbreak attacks to manipulate LLMs’ judgment over pluralistic values by using a morality dataset with 10.4K instances. |
| Outcome: | The proposed method exploits the persuasion abilities of LLMs to produce moral content over pluralistic values. |
Similar Papers
Large Language Models Are Involuntary Truth-Tellers: Exploiting Fallacy Failure for Jailbreak Attacks (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing research has shown that large language models have difficulty discerning the veracity of their intrinsic answers. |
| Approach: | They propose a jailbreak attack method that generates an aligned language model for malicious output. |
| Outcome: | The proposed method achieves competitive performance with more harmful outputs. |
Safety Guardrails of Large Language Models Are Vulnerable to Value-Driven Adversarial Prompting (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing jailbreak attacks against large language models (LLMs) can be divided into white-box attacks and black-box attack. |
| Approach: | They propose a value-driven jailbreak attack that exploits the phenomenon that large language models agree with humans to induce LLMs to affirm the moral value of harmful tasks. |
| Outcome: | Extensive experiments on five state-of-the-art (SOTA) LLMs show the value-driven jailbreak attack achieves an average attack success rate (ASR) of 91.8% on JailbreakBench and 95.2% on the AdvBench subset. |
Shaping the Safety Boundaries: Understanding and Defending Against Jailbreaks in Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Understanding how jailbreaking works remains limited, hindering the development of effective defense strategies. |
| Approach: | They propose a new mechanism that adaptively constrains activations within the safety boundary and propose 'Activation Boundary Defense' to enhance its effectiveness. |
| Outcome: | The proposed defense achieves an average Defense Success Rate (DSR) of over 98% against various jailbreak attacks, with less than 2% impact on the model’s general capabilities. |
CAVGAN: Unifying Jailbreak and Defense of LLMs via Generative Adversarial Attacks on their Internal Representations (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have isolated LLM jailbreak attacks and defenses . a new framework combines attack and defense to protect against malicious queries . |
| Approach: | They propose a framework that combines attack and defense to protect the Large Language Model (LLM) by embedding harmful problems into the safe area. |
| Outcome: | The proposed framework achieves an average jailbreak success rate of 88.85% across three popular LLMs while the defense success rate reaches an average of 84.17%. |
A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated capabilities for generating content that could be deemed harmful. |
| Approach: | They conduct a comprehensive analysis of existing studies on jailbreaking LLMs and their defense techniques. |
| Outcome: | The proposed techniques underperform existing white-box attacks and include special tokens significantly affects the likelihood of successful attacks. |
How Alignment and Jailbreak Work: Explain LLM Safety through Intermediate Hidden States (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) rely on safety alignment to avoid malicious user inputs. |
| Approach: | They employ weak classifiers to explain LLM safety through the intermediate hidden states. |
| Outcome: | The proposed model can identify malicious and normal inputs and detect malicious ones without jailbreak. |
Diversity Helps Jailbreak Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for jailbreaking large language models rely on laborious human engineering and whitebox access to model internals. |
| Approach: | They propose a method that instructs large language models to deviate from prior context and generate harmful outputs by instructing them to deviat from previous attacks. |
| Outcome: | The proposed method achieves a 62.83% higher success rate in compromising ten leading chatbots, while using only 12.9% of the queries. |
Understanding Jailbreak Success: A Study of Latent Space Dynamics in Large Language Models (2026.eacl-long)
Copied to clipboard
| Challenge: | Emerging jailbreaking techniques can still elicit unsafe outputs, presenting an ongoing challenge for model alignment. |
| Approach: | They propose to extract a jailbreak vector from a single class of jailbreaks that works to mitigate jailbreak effectiveness from other, semantically-dissimilar classes. |
| Outcome: | The proposed jailbreak vectors show that they reduce harmfulness in most models, and that they are similar in geometric and effect similarity. |
Unleashing the Unseen: Harnessing Benign Datasets for Jailbreaking Large Language Models (2026.findings-eacl)
Copied to clipboard
| Challenge: | Despite significant efforts in safety alignment, large language models (LLMs) such as GPT-4 and LLaMA 3 remain vulnerable to jailbreak attacks that can induce harmful behaviors. |
| Approach: | They propose a feature extraction method to extract sample-agnostic features from benign datasets in the form of adversarial suffixes and propose 'suffix maybe features' they show that adversarials generated from jailbreak attacks may contain meaningful features, i.e. appending the same suffix to different prompts results in responses exhibiting specific characteristics. |
| Outcome: | The proposed method extracts sample-agnostic features from benign datasets and shows that they may contain meaningful features. |
Confusion is the Final Barrier: Rethinking Jailbreak Evaluation and Investigating the Real Misuse Threat of LLMs (2025.findings-emnlp)
Copied to clipboard
Yu Yan, Sheng Sun, Zhe Wang, Yijun Lin, Zenghao Duan, Zhifei Zheng, Min Liu, Zhiyi Yin, Jianping Zhang
| Challenge: | Large Language Models have been developed to deal with real-world crimes, but it remains unclear whether they internalize authentic knowledge or are forced to simulate toxic language patterns. |
| Approach: | They construct knowledge-intensive Q&A to investigate misuse threats of Large Language Models in terms of dangerous knowledge possession, harmful task planning utility, and harmfulness judgment robustness. |
| Outcome: | The findings raise concerns that jailbreak success is often attributable to a hallucination loop between jailbroken LLM and judger LLM . |