ToxiCraft: A Novel Framework for Synthetic Generation of Harmful Information (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models for detecting harmful content lack diversity and quality of datasets. |
| Approach: | They propose a framework for synthesizing toxic information from social media datasets . their framework generates a wide variety of synthetic, yet remarkably realistic, examples of toxic information . |
| Outcome: | The proposed framework can generate a wide variety of synthetic, yet remarkably realistic, examples of toxic information. |
Similar Papers
Towards Building a Robust Toxicity Predictor (2023.acl-industry)
Copied to clipboard
| Challenge: | Recent studies have focused on robustness of toxicity language predictors, but this is problematic for real-world toxicity detection. |
| Approach: | They propose a novel adversarial attack that exploits greedy search strategies to fool toxic text classifiers. |
| Outcome: | The proposed attack can detect weaker toxicity language detectors even against unseen attacks. |
ToVo: Toxicity Taxonomy via Voting (2025.findings-naacl)
Copied to clipboard
Tinh Son Luong, Thanh-Thien Le, Thang Viet Doan, Linh Ngo Van, Thien Huu Nguyen, Nguyen Thi Ngoc Diep
| Challenge: | Existing toxic content detection models face limitations due to the closed-source nature of training data and the paucity of explanations for their evaluation mechanism. |
| Approach: | They propose a mechanism that integrates voting and chain-of-thought processes to produce a high-quality open-source dataset for toxic content detection. |
| Outcome: | The proposed model improves transparency and customizability while facilitating better fine-tuning for specific use cases. |
Explaining Toxic Text via Knowledge Enhanced Text Generation (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing work on toxic speech classification relies on generic and repetitive explanations . elucidating toxic speech can help with downstream tasks such as debiasing . |
| Approach: | They propose a knowledge-informed encoder-decoder framework to generate toxic text explanations . they use multiple knowledge sources to generate detailed explanations of toxic text . |
| Outcome: | The proposed model outperforms state-of-the-art models significantly in generating toxic explanations . the proposed model can generate detailed explanations of toxic speech compared to baselines compared with baseline models . |
Beyond Static Benchmarks: Synthesizing Harmful Content via Persona-based Simulation for Robust Evaluation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing static benchmarks for harmful content detection face limitations in scalability and diversity. |
| Approach: | They propose a framework for synthesizing harmful content using persona-guided large language model agents. |
| Outcome: | The proposed framework achieves a high success rate in harmful generation tests across multiple detection systems. |
A little goes a long way: Improving toxic language classification despite data scarcity (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for toxic language classification have not been thoroughly explored. |
| Approach: | They propose to use data augmentation to generate new synthetic data from labeled seed datasets to improve toxic language classification. |
| Outcome: | The proposed techniques perform well on very scarce toxic language datasets while performing worse on shallower models. |
Don’t Augment, Rewrite? Assessing Abusive Language Detection with Synthetic Data (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing datasets for abusive language detection and content moderation are limited by regulatory bodies and social media platforms. |
| Approach: | They propose to replace existing datasets in English with synthetic data by rewriting original texts with an instruction-based generative model. |
| Outcome: | The proposed model improves performance in cross-dataset training. |
Mitigating Data Poisoning in Text Classification with Differential Privacy (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Data poisoning attacks can plant a backdoor in a model by injecting poisoned examples into training data, causing the model to misclassify test instances which include a specific pattern. |
| Approach: | They propose a generic defence mechanism that makes training robust to poisoning attacks by smoothing the gradient from each training example. |
| Outcome: | The proposed method is highly effective in mitigating, or even eliminating, poisoning attacks on text classification, with only a small cost in predictive accuracy. |
TaeBench: Improving Quality of Toxic Adversarial Examples (2025.naacl-industry)
Copied to clipboard
| Challenge: | Existing adversarial examples generate invalid or ambiguous examples that fool the systems into wrong detection. |
| Approach: | They propose an annotation pipeline for quality control of generated toxic adversarial examples (TAE) they use model-based automated annotation and human-based quality verification to assess quality requirements of a TAE dataset. |
| Outcome: | The proposed pipeline can transfer-attack SOTA toxicity content moderation models and services with adversarial training. |
Fortifying Toxic Speech Detectors Against Veiled Toxicity (2020.emnlp-main)
Copied to clipboard
| Challenge: | Modern toxic speech detectors are incompetent in recognizing disguised offensive language, such as adversarial attacks that deliberately avoid known toxic lexicons. |
| Approach: | They propose a framework that fortifies existing toxic speech detectors without a large labeled corpus of veiled toxicity. |
| Outcome: | The proposed framework is aimed at fortifying existing toxic speech detectors without a large labeled corpus of disguised offensive language. |
Mitigating Societal Harms in Large Language Models (2023.emnlp-tutorial)
Copied to clipboard
| Challenge: | Recent studies have highlighted societal harms that can be caused by language generation models deployed in the wild. |
| Approach: | They propose to use a typology of technical approaches to mitigating harms of language generation models to provide an overview of potential social issues in language generation including toxicity, social biases, misinformation, factual inconsistency, and privacy violations. |
| Outcome: | The proposed typology addresses toxicity, biases, misinformation, factual inconsistency, and privacy violations in language generation models. |