Evaluation of LLM Vulnerabilities to Being Misused for Personalized Disinformation Generation (2025.acl-long)
Copied to clipboard
Aneta Zugecova, Dominik Macko, Ivan Srba, Robert Moro, Jakub Kopál, Katarína Marcinčinová, Matúš Mesarčík
| Challenge: | Recent large language models generate disinformation news articles following predefined narratives . personalization and disinformation abilities of LLMs have not been studied . |
| Approach: | They evaluate the personalization and disinformation abilities of large language models . they find personalization reduces the safety-filter activations, thus effectively functioning as a jailbreak . |
| Outcome: | The proposed model generates disinformation news articles in english with the lowest quality of personalization. |
Similar Papers
On the Risk of Misinformation Pollution with Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study demonstrates that large language models can be misused for generating credible-sounding misinformation . however, the ability to produce credible text raises concerns regarding their potential misuse . |
| Approach: | They propose three defense strategies to mitigate misinformation generated by Large Language Models . they propose a threat model and simulate potential misuse scenarios . |
| Outcome: | The proposed defense strategies have shown promising results, albeit with costs. |
Confusion is the Final Barrier: Rethinking Jailbreak Evaluation and Investigating the Real Misuse Threat of LLMs (2025.findings-emnlp)
Copied to clipboard
Yu Yan, Sheng Sun, Zhe Wang, Yijun Lin, Zenghao Duan, Zhifei Zheng, Min Liu, Zhiyi Yin, Jianping Zhang
| Challenge: | Large Language Models have been developed to deal with real-world crimes, but it remains unclear whether they internalize authentic knowledge or are forced to simulate toxic language patterns. |
| Approach: | They construct knowledge-intensive Q&A to investigate misuse threats of Large Language Models in terms of dangerous knowledge possession, harmful task planning utility, and harmfulness judgment robustness. |
| Outcome: | The findings raise concerns that jailbreak success is often attributable to a hallucination loop between jailbroken LLM and judger LLM . |
How to Protect Yourself from 5G Radiation? Investigating LLM Responses to Implicit Misinformation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Current studies evaluate LLMs on explicit false statements, overlooking how misinformation manifests subtly as unchallenged premises in real-world interactions. |
| Approach: | They propose to use EchoMist to analyze implicit misinformation from diverse sources . they also investigate two mitigation methods to enhance LLMs’ capability to counter implicit mis information. |
| Outcome: | The proposed model fails to detect false premises and generate counterfactual explanations. |
Intrinsic Model Weaknesses: How Priming Attacks Unveil Vulnerabilities in Large Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have significant impact on various industries and societal functions due to advanced instruction-following capabilities. |
| Approach: | They developed and tested novel attack strategies on popular LLMs to expose their vulnerabilities in generating harmful content. |
| Outcome: | The proposed attacks achieved an ASR of 100% on open-source models, including Meta’s Llama-3.2, Google’s Gemma-2, Mistral’s Mistral-NeMo, Falcon’s Falcon-mamba, Apple’s DCLM, Microsoft’s Phi3, and Qwen’s Qwend2.5, among others. |
A Survey on Detection of LLMs-Generated Content (2024.findings-emnlp)
Copied to clipboard
Xianjun Yang, Liangming Pan, Xuandong Zhao, Haifeng Chen, Linda Petzold, William Yang Wang, Wei Cheng
| Challenge: | Recent advances in large language models have led to an increase in synthetic content generation . the ability to detect LLMs-generated content has become of paramount importance . |
| Approach: | They propose to provide a detailed overview of existing detection strategies and benchmarks, scrutinizing their differences and advocating for more adaptable and robust models to enhance detection accuracy. |
| Outcome: | The proposed model will be able to detect human-written content in real time. |
Combating Security and Privacy Issues in the Era of Large Language Models (2024.naacl-tutorials)
Copied to clipboard
| Challenge: | a tutorial aims to provide a summary of risks and vulnerabilities in large language models . a number of studies have focused on security, privacy and copyright aspects of LLMs . |
| Approach: | This tutorial seeks to provide a systematic summary of risks and vulnerabilities in large language models . authors will discuss security, privacy and copyright aspects of LLMs . |
| Outcome: | This tutorial aims to provide a systematic summary of risks and vulnerabilities in large language models . it will also outline emerging challenges in security, privacy and reliability of LLMs . |
LLMs Caught in the Crossfire: Malware Requests and Jailbreak Challenges (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have a high vulnerability to jailbreak attacks that leverage crafted prompts to generate malicious outputs. |
| Approach: | They propose to use large language models to test their security against jailbreak attacks that leverage crafted prompts to generate malicious outputs. |
| Outcome: | The proposed model is based on 320 manually crafted malicious code generation requirements, covering 11 jailbreak methods and 29 code functionality categories. |
Exploring Safety-Utility Trade-Offs in Personalized Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Prior studies have shown that large language models can exhibit bias against specific demographic groups and engage in the generation of stereotypical responses. |
| Approach: | They propose a framework to evaluate LLM performance along two axes: safety and utility. |
| Outcome: | The proposed framework evaluates the performance of LLMs along two axes: safety and utility. |
Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreaks (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods to jailbreak large language models have been poorly studied . a recent study showed that non-expert users can jailbreak LLMs by manipulating their prompts . |
| Approach: | They propose a formalism and a taxonomy of known (and possible) jailbreaks . they propose generating a dataset of model outputs across 3700 jailbreak prompts a 'prompt' attack is a new attack popularly categorized as "prompting injection attacks" |
| Outcome: | The proposed model exploits 3700 jailbreak prompts over 4 tasks to analyze their effectiveness . authors show that the model can learn to perform a new task on unseen examples . |
Revisiting Jailbreaking for Large Language Models: A Representation Engineering Perspective (2025.coling-main)
Copied to clipboard
Tianlong Li, Zhenghua Wang, Wenhao Liu, Muling Wu, Shihan Dou, Changze Lv, Xiaohua Wang, Xiaoqing Zheng, Xuanjing Huang
| Challenge: | Recent surge in jailbreaking attacks has revealed significant vulnerabilities in Large Language Models (LLMs) however, limited research into the underlying mechanisms that make LLMs vulnerable to such attacks has been conducted. |
| Approach: | They propose that LLMs' self-safeguarding capability is linked to specific activity patterns within their representation space. |
| Outcome: | The proposed models can be detected with a few pairs of contrastive queries, and the robustness can be manipulated by weakening or strengthening these patterns. |