| Challenge: | Existing methods for large language models focus on post-training stage or inference time, while few tackle the source of toxicity, namely, the dataset itself. |
| Approach: | They propose to localize and rewrite toxic spans in raw corpora with SoCD, which guides an LLM to localized and preserving semantics while preserving toxicity. |
| Outcome: | The proposed method reduces TP from 0.42 to 0.18 and Expected Maximum Toxicity (EMT) from 0.43 to 0.20 on three LLMs. |
Similar Papers
Adaptive Detoxification: Safeguarding General Capabilities of LLMs through Toxicity-Aware Knowledge Editing (2025.findings-acl)
Copied to clipboard
Yifan Lu, Jing Li, Yigeng Zhou, Yihui Zhang, Wenya Wang, Xiucheng Li, Meishan Zhang, Fangming Liu, Jun Yu, Min Zhang
| Challenge: | Existing knowledge editing methods for large language models (LLMs) suffer from over-editing, where detoxified models reject legitimate queries, compromising overall performance. |
| Approach: | They propose a toxicity-aware knowledge editing approach that dynamically detects toxic activation patterns during forward propagation and then routes computations through adaptive inter-layer pathways to mitigate toxicity effectively. |
| Outcome: | The proposed method outperforms existing methods on large language models and enhances the SafeEdit benchmark. |
Text Detoxification: Data Efficiency, Semantic Preservation and Model Generalization (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for detoxification of text often rely on manually annotated data . xiangli: "detoxification of texts is a powerful way to remove toxic content" |
| Approach: | They propose a reinforcement learning framework that optimizes detoxification and semantic preservation without annotating large amounts of data. |
| Outcome: | The proposed method overcomes major limitations and surpasses humanannotated references across multiple benchmarks. |
DSCD: Large Language Model Detoxification with Self-Constrained Decoding (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for decoding large language models (LLMs) are based on external constraints and require additional resource overhead and loss of generation fluency. |
| Approach: | They propose a method for LLMs detoxification without parameter fine-tuning that strengthens the inner token distribution while weakening that of hallucination and toxic layer during output generation. |
| Outcome: | Extensive experiments on open-source LLMs and public datasets demonstrate DSCD's state-of-the-art (SOTA) performance in detoxification and generation fluency, with superior efficiency compared to existing methods. |
Detoxifying Large Language Models via Knowledge Editing (2024.acl-long)
Copied to clipboard
Mengru Wang, Ningyu Zhang, Ziwen Xu, Zekun Xi, Shumin Deng, Yunzhi Yao, Qishen Zhang, Linyi Yang, Jindong Wang, Huajun Chen
| Challenge: | Existing methods to detoxify Large Language Models (LLMs) are limiting, but knowledge editing can be effective. |
| Approach: | They propose a baseline method to detoxify Large Language Models (LLMs) they propose supervised fine-tuning and reinforcement learning from human feedback (RLHF) |
| Outcome: | The proposed method reduces toxicity of large language models with one instance of tuning . it reduces the toxicity, while minimizing the toxins, the authors show . |
CMD: a framework for Context-aware Model self-Detoxification (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods of text detoxification fail to achieve a decent balance between effectiveness and generation quality. |
| Approach: | They propose a text detoxification framework that pays attention to both context and detoxification process. |
| Outcome: | Experiments on various LLMs show that the proposed framework can yield the best performance compared to baselines. |
MICo: Preventative Detoxification of Large Language Models through Inhibition Control (2024.findings-naacl)
Copied to clipboard
Roy Siegelmann, Ninareh Mehrabi, Palash Goyal, Prasoon Goyal, Lisa Bauer, Jwala Dhamala, Aram Galstyan, Rahul Gupta, Reza Ghanadan
| Challenge: | Large Language Models (LLMs) have a tendency to devolve into toxic degeneration . model may classify prompts as toxic or non-toxic and categorically refuse to respond to those deemed toxic. |
| Approach: | They propose a mechanism for LLM detoxification by labeling acceptable and unacceptable examples and including a corresponding acceptable rewrite with every unacceptable example. |
| Outcome: | The proposed model improves on the baseline model and shows that it detects and rewrites toxic and harmful examples. |
Contrastive Perplexity for Controlled Generation: An Application in Detoxifying Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to generate toxic content by large language models are based on pipelines . current approaches focus on preserving performance while effectively mitigating toxicity . |
| Approach: | They propose a framework for implicit knowledge editing and controlled text generation by using hard negatives. |
| Outcome: | The proposed framework significantly reduces toxic generation while maintaining strong performance on downstream tasks. |
Projecting Out the Malice: A Global Subspace Approach to LLM Detoxification (2026.acl-long)
Copied to clipboard
Zenghao Duan, Zhiyi Yin, Zhichao Shi, Liang Pang, Shaoling Jing, Zihe Huang, Jiayi Wu, Yu Yan, Jingcheng Deng, Huawei Shen, Xueqi Cheng
| Challenge: | Large language models (LLMs) exhibit exceptional performance but pose inherent risks of generating toxic content. |
| Approach: | They propose a method that removes toxic subspaces from FFN parameters . they propose to use a lightweight method to eliminate toxic subespaces . |
| Outcome: | The proposed method achieves SOTA detoxification while preserving general capabilities without large-scale retraining. |
Detoxifying Large Language Models via the Diversity of Toxic Samples (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for analyzing and utilizing toxic samples are limited . current methods fail to fully harness their potential . |
| Approach: | They propose a diverse detoxification framework that leverages toxic samples' diversity . they propose MPSG strategy and SC-DPO approach to elicit personalized toxic responses . |
| Outcome: | The proposed framework could be used to optimize large language models for user safety . it incorporates two components: MPSG strategy and SC-DPO approach . |
Cross-Lingual Transfer of Debiasing and Detoxification in Multilingual LLMs: An Extensive Investigation (2025.findings-acl)
Copied to clipboard
| Challenge: | Prior work has shown that finetuning on specialized datasets can mitigate this behavior, and doing so in English can transfer to other languages. |
| Approach: | They propose to fine tune generative large language models to provide safe responses to harmful user input and to use direct preference optimization to mitigate toxicity. |
| Outcome: | The proposed models show that finetuning on specialized datasets reduces biases but also produces fluent and diverse text in non-English languages. |