| Challenge: | Existing safety benchmarks and red teaming prompts fail to capture implicit toxicity in complex real-life advice-seeking scenarios. |
| Approach: | They propose a dataset designed for identifying implicit toxicity within advice-seeking scenarios. |
| Outcome: | The proposed dataset matches or surpasses the zero-shot performance of large language models in toxicity classification tasks. |
Similar Papers
MuTox: Universal MUltilingual Audio-based TOXicity Dataset and Zero-shot Detector (2024.findings-acl)
Copied to clipboard
Marta Costa-jussà, Mariano Meglioli, Pierre Andrews, David Dale, Prangthip Hansanti, Elahe Kalbassi, Alexandre Mourachko, Christophe Ropers, Carleigh Wood
| Challenge: | Existing studies on text-based toxicity detection for other languages are limited, especially for languages other than English. |
| Approach: | They propose a multilingual audio-based toxicity classifier which covers 14 different linguistic families and a dataset of 20,000 audio utterances for English and Spanish. |
| Outcome: | The new classifier improves F1-Score by an average of 100% when compared to existing wordlist-based classifiers. |
Unveiling the Implicit Toxicity in Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies focus on probing toxic outputs that can be easily detected with existing toxicity classifiers, but LLMs can generate diverse implicit toxic output that are difficult to detect via simply zero-shot prompting. |
| Approach: | They propose a reinforcement learning based attacking method to induce the implicit toxic outputs in large language models by fine-tuning toxicity classifiers. |
| Outcome: | The proposed method generates implicit toxic outputs that are difficult to detect via zero-shot prompting on five widely-adopted toxicity classifiers. |
Enhancing LLM-based Hatred and Toxicity Detection with Meta-Toxic Knowledge Graph (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to address toxicity issues with large language models are inadequate . lack of domain-specific knowledge leads to false negatives and excessive sensitivity to toxic speech limits freedom of speech. |
| Approach: | They propose a method that leverages graph search on a meta-toxic knowledge graph to enhance hatred and toxicity detection. |
| Outcome: | The proposed method lowers false positive rate and improves toxicity detection performance in out-of-domain scenarios. |
GameTox: A Comprehensive Dataset and Analysis for Enhanced Toxicity Detection in Online Gaming Communities (2025.naacl-short)
Copied to clipboard
| Challenge: | Existing methods to detect toxic behavior in online gaming environments are limited by utterance-level annotation. |
| Approach: | They propose to annotate game chat utterances for toxicity detection through intent classification and slot filling. |
| Outcome: | The proposed model improves the detection of toxic speech in online gaming environments and reveals limitations of current models. |
On the Role of Speech Data in Reducing Toxicity Detection Bias (2025.naacl-long)
Copied to clipboard
Samuel Bell, Mariano Coria Meglioli, Megan Richards, Eduardo Sánchez, Christophe Ropers, Skyler Wang, Adina Williams, Levent Sagun, Marta R. Costa-jussà
| Challenge: | Text toxicity detection systems produce disproportionate rates of false positives on demographic groups . toxicity classification systems often misinterpret benign group mentions as toxic . |
| Approach: | They use group annotations to compare text-based and speech-based toxicity detection systems. |
| Outcome: | The results show that access to speech data supports reduced bias against group mentions . the authors recommend improving classifiers, rather than transcription pipelines if possible . |
Can Large Language Models Identify Implicit Suicidal Ideation? An Empirical Evaluation (2025.findings-emnlp)
Copied to clipboard
Tong Li, Shu Yang, Junchao Wu, Jiyao Wei, Lijie Hu, Mengdi Li, Derek F. Wong, Joshua R. Oltmanns, Di Wang
| Challenge: | Existing data on suicidal ideation in private conversations are limited . a new dataset of 1,200 test cases is presented to address this gap . |
| Approach: | They propose a dataset of 1,200 test cases simulating implicit suicidal ideation in private contexts. |
| Outcome: | The proposed dataset includes 1,200 test cases simulating implicit suicidal ideation in dialogue scenarios. |
Detoxifying Large Language Models via the Diversity of Toxic Samples (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for analyzing and utilizing toxic samples are limited . current methods fail to fully harness their potential . |
| Approach: | They propose a diverse detoxification framework that leverages toxic samples' diversity . they propose MPSG strategy and SC-DPO approach to elicit personalized toxic responses . |
| Outcome: | The proposed framework could be used to optimize large language models for user safety . it incorporates two components: MPSG strategy and SC-DPO approach . |
Realistic Evaluation of Toxicity in Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | a large amount of data exposes large language models to toxicity and bias . prompt engineering can be easily bypassed with minimal prompt engineering. |
| Approach: | They propose a dataset that uses manually crafted prompts to nullify protective layers of large language models. |
| Outcome: | The proposed dataset shows that prompts can nullify protective layers of large language models. |
New Terms, New Toxicity: Consensus-based Chinese Neologism Toxicity Detection via Search-Augmented LLMs (2026.acl-long)
Copied to clipboard
Shiyao Cui, QingLin Zhang, Di Wang, Yida Lu, Zhexin Zhang, Jinhua Gao, Jinglin Yang, Min He, Han Qiu, Minlie Huang
| Challenge: | Neologisms can foster new linguistic consensus by stabilizing shared meanings and usage in common communicative norms. |
| Approach: | They propose a taxonomy that captures the origins and consensus-verification criteria of toxic neologisms . they propose 'SeTox' framework that integrates real-time web context for naeologim detection . |
| Outcome: | The proposed framework outperforms large-scale models in detecting neologism toxicity. |
Tox-BART: Leveraging Toxicity Attributes for Explanation Generation of Implicit Hate Speech (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing language models to generate implicit hate explanations are lacking in many fields. |
| Approach: | They propose to use language models to generate explicit hate posts to make it clear . they find that simpler models incorporating external toxicity signals outperform KG-infused models . |
| Outcome: | The proposed setup produces more precise explanations than zero-shot GPT-3.5, highlighting the intricate nature of the task. |