Jailbreak Open-Sourced Large Language Models via Enforced Decoding (2024.acl-long)
Copied to clipboard
Hangfan Zhang, Zhimeng Guo, Huaisheng Zhu, Bochuan Cao, Lu Lin, Jinyuan Jia, Jinghui Chen, Dinghao Wu
| Challenge: | Existing studies show that Large Language Models can be misused to generate undesired content. |
| Approach: | They propose to use large language models to manipulate the generation process to generate undesired content without heavy computations or prompt designs. |
| Outcome: | The proposed method shows that open-sourced large language models could be misused to generate undesired content without heavy computations or prompt designs. |
Similar Papers
On Weaponization-Resistant Large Language Models with Prospect Theoretic Alignment (2025.coling-main)
Copied to clipboard
| Challenge: | Existing safeguards for large language models are inadequate for open-weight models as minimal fine-tuning can bypass them. |
| Approach: | They propose a framework that prioritizes maximizing generative utility rather than a singular optimization metric and integrates prospect theory into LLM training to strengthen LLMs against misuse and weaponization. |
| Outcome: | The proposed framework strengthens LLMs against misuse and weaponization while maintaining high performance even after extensive fine-tuning. |
Defending Against Alignment-Breaking Attacks via Robustly Aligned LLM (2024.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have made significant advancements but can be misused to generate harmful content. |
| Approach: | They propose a Robustly Aligned LLM to defend against alignment-breaking attacks by retraining existing LLMs and using adversarial or handcrafted jailbreaking prompts. |
| Outcome: | The proposed model reduces attack success rates from nearly 100% to around 10% or less. |
On the Vulnerability of Safety Alignment in Open-Access LLMs (2024.findings-acl)
Copied to clipboard
Jingwei Yi, Rui Ye, Qisi Chen, Bin Zhu, Siheng Chen, Defu Lian, Guangzhong Sun, Xing Xie, Fangzhao Wu
| Challenge: | Large language models (LLMs) are susceptible to malicious exploitation, but are often rejected and limited harmfulness is limited. |
| Approach: | They propose two types of reverse alignment techniques: reverse supervised fine-tuning (RSFT) and reverse preference optimization (RPO). |
| Outcome: | The proposed methods can significantly enhance the success rate and harmfulness of jailbreak attacks, but they face high rejection rates and limited harmfulness. |
Improving the OOD Performance of Closed-Source LLMs on NLI Through Strategic Data Selection (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods to improve robustness require changing the fine-tuning process or large-scale data augmentation, which are infeasible or cost prohibitive for closed-source models. |
| Approach: | They propose to prioritize more complex examples or replace existing training examples with LLM-generated data to improve performance on OOD NLI datasets. |
| Outcome: | The proposed methods improve performance on difficult OOD datasets while training with synthetic data leads to substantial improvements on easier OOD data. |
Large Language Models Are Involuntary Truth-Tellers: Exploiting Fallacy Failure for Jailbreak Attacks (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing research has shown that large language models have difficulty discerning the veracity of their intrinsic answers. |
| Approach: | They propose a jailbreak attack method that generates an aligned language model for malicious output. |
| Outcome: | The proposed method achieves competitive performance with more harmful outputs. |
A Survey on Training-free Alignment of Large Language Models (2025.findings-emnlp)
Copied to clipboard
Birong Pan, Yongqi Li, Weiyu Zhang, Wenpeng Lu, Mayi Xu, Shen Zhou, Yuanyuan Zhu, Ming Zhong, Tieyun Qian
| Challenge: | a survey of large language models (LLMs) aims to ensure outputs adhere to human values, ethical standards, and legal norms. |
| Approach: | They present the first systematic review of TF alignment methods . they categorize them by stages of pre-decoding, in-decoder and post-decoration . |
| Outcome: | The proposed methods are based on training-free (TF) alignment techniques . they are able to be used in open-source and closed-source environments without retraining . |
Unlocking Anticipatory Text Generation: A Constrained Approach for Large Language Models Decoding (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models have shown a powerful ability for text generation, but undesired behaviors such as toxicity and hallucinations can manifest. |
| Approach: | They propose to formalize text generation as a future-constrained generation problem to minimize undesirable behaviors and enforce faithfulness to instructions. |
| Outcome: | The proposed approach is effective across three tasks, including keyword-constrained generation, toxicity reduction, and factual correctness in question-answering. |
Aligning Large Language Models through Synthetic Feedback (2023.emnlp-main)
Copied to clipboard
| Challenge: | Currently, alignment learning requires significant human demonstrations and feedback from proprietary LLMs such as ChatGPT. |
| Approach: | They propose a framework that uses synthetic feedback to align large language models to human values without extensive human annotations and proprietary LLMs. |
| Outcome: | The proposed model outperforms open-source models on human-annotated demonstrations in alignment benchmarks. |
Removing RLHF Protections in GPT-4 via Fine-Tuning (2024.naacl-short)
Copied to clipboard
| Challenge: | Large language models (LLMs) have increased in their capabilities, which increases their potential for dual use. |
| Approach: | They show that fine-tuning can remove RLHFprotections with as few as 340 examples and a 95% success rate. |
| Outcome: | The proposed method removes RLHFprotections with as few as 340 examples and a 95% success rate on non-censored outputs. |
Rendering Data Unlearnable by Exploiting LLM Alignment Mechanisms (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly trained on massive, heterogeneous text corpora, raising serious concerns about the unauthorised use of proprietary or personal data during model training. |
| Approach: | They propose a data-level defence that renders text unlearnable to LLMs by injecting carefully designed alignment-triggering disclaimers into the models' alignment mechanisms. |
| Outcome: | The proposed approach exploits the models’ alignment mechanisms to prevent effective learning. |