| Challenge: | Existing mechanisms to restrict behavior of language models (LMs) are vulnerable to misuse and misalignment. |
| Approach: | They propose a mechanism to restrict specific behaviors in language models by exploiting hardware properties. |
| Outcome: | The proposed mechanism can be applied to trigger overflows for specific behaviors or target hardware malfunctions. |
Similar Papers
Revisiting Jailbreaking for Large Language Models: A Representation Engineering Perspective (2025.coling-main)
Copied to clipboard
Tianlong Li, Zhenghua Wang, Wenhao Liu, Muling Wu, Shihan Dou, Changze Lv, Xiaohua Wang, Xiaoqing Zheng, Xuanjing Huang
| Challenge: | Recent surge in jailbreaking attacks has revealed significant vulnerabilities in Large Language Models (LLMs) however, limited research into the underlying mechanisms that make LLMs vulnerable to such attacks has been conducted. |
| Approach: | They propose that LLMs' self-safeguarding capability is linked to specific activity patterns within their representation space. |
| Outcome: | The proposed models can be detected with a few pairs of contrastive queries, and the robustness can be manipulated by weakening or strengthening these patterns. |
SELF-GUARD: Empower the LLM to Safeguard Itself (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent studies have investigated methods to improve the safety of large language models (LLMs) safety training involves fine-tuning the LLM with adversarial samples, which activate the LRM’s capabilities against jailbreak. |
| Approach: | They propose a safety training approach that integrates safety training and safeguards to train the LLM to perform harmfulness detection on its own outputs. |
| Outcome: | The proposed method reduces harmful output and adds a [harmful] or [harmless] tag to the end of the LLM's response. |
Root Defense Strategies: Ensuring Safety of LLM at the Decoding Level (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods to detect harmful outputs from prefill-level lacks utilization of the model’s decoding outputs, leading to relatively lower effectiveness and robustness. |
| Approach: | They propose a robust decoding mechanism that corrects harmful queries directly rather than rejecting them outright. |
| Outcome: | The proposed model improves model security without compromising reasoning speed. |
Diversity Helps Jailbreak Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for jailbreaking large language models rely on laborious human engineering and whitebox access to model internals. |
| Approach: | They propose a method that instructs large language models to deviate from prior context and generate harmful outputs by instructing them to deviat from previous attacks. |
| Outcome: | The proposed method achieves a 62.83% higher success rate in compromising ten leading chatbots, while using only 12.9% of the queries. |
Beyond the Tip of Efficiency: Uncovering the Submerged Threats of Jailbreak Attacks in Small Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Small language models (SLMs) have become increasingly prominent in the deployment on edge devices due to their high efficiency and low computational cost. |
| Approach: | They evaluate the security performance of 13 state-of-the-art small language models under various jailbreak attacks. |
| Outcome: | The proposed methods demonstrate that SLMs are quite susceptible to jailbreak attacks and some are even vulnerable to harmful prompts. |
Why Not Act on What You Know? Unleashing Safety Potential of LLMs via Self-Aware Guard Enhancement (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated impressive capabilities across various tasks but are vulnerable to meticulously crafted jailbreak attacks. |
| Approach: | They propose a training-free defense strategy to align LLMs’ strong safety discrimination performance with their relatively weaker safety generation ability. |
| Outcome: | The proposed strategy achieves an average 99% success rate against numerous complex and covert jailbreak methods while maintaining helpfulness on general benchmarks. |
A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated capabilities for generating content that could be deemed harmful. |
| Approach: | They conduct a comprehensive analysis of existing studies on jailbreaking LLMs and their defense techniques. |
| Outcome: | The proposed techniques underperform existing white-box attacks and include special tokens significantly affects the likelihood of successful attacks. |
Combating Security and Privacy Issues in the Era of Large Language Models (2024.naacl-tutorials)
Copied to clipboard
| Challenge: | a tutorial aims to provide a summary of risks and vulnerabilities in large language models . a number of studies have focused on security, privacy and copyright aspects of LLMs . |
| Approach: | This tutorial seeks to provide a systematic summary of risks and vulnerabilities in large language models . authors will discuss security, privacy and copyright aspects of LLMs . |
| Outcome: | This tutorial aims to provide a systematic summary of risks and vulnerabilities in large language models . it will also outline emerging challenges in security, privacy and reliability of LLMs . |
DELMAN: Dynamic Defense Against Large Language Model Jailbreaking with Model Editing (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing safety mechanisms for Large Language Models (LLMs) are inadequate to protect against jailbreak attacks, resulting in performance degradation on general tasks. |
| Approach: | They propose a method that directly updates a minimal set of relevant parameters to neutralize harmful behaviors while preserving the model’s utility. |
| Outcome: | The proposed model outperforms baseline methods in mitigating jailbreak attacks while preserving the model’s utility. |
Shaping the Safety Boundaries: Understanding and Defending Against Jailbreaks in Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Understanding how jailbreaking works remains limited, hindering the development of effective defense strategies. |
| Approach: | They propose a new mechanism that adaptively constrains activations within the safety boundary and propose 'Activation Boundary Defense' to enhance its effectiveness. |
| Outcome: | The proposed defense achieves an average Defense Success Rate (DSR) of over 98% against various jailbreak attacks, with less than 2% impact on the model’s general capabilities. |