| Challenge: | Existing ‘toxicity’ detection datasets and models ignore the context of the posts, implicitly assuming that comments may be judged independently. |
| Approach: | They limit the notion of context to the previous post in the thread and the discussion title and focus on how it affects human judgement. |
| Outcome: | The proposed model can amplify or mitigate perceived toxicity of posts and a small but significant subset of manually labeled posts end up having the opposite toxicity labels if the annotators are not provided with context. |
Similar Papers
On the Challenges of Using Black-Box APIs for Toxicity Evaluation in Research (2023.emnlp-main)
Copied to clipboard
| Challenge: | Perception of toxicity evolves over time and differs between geographies and cultural backgrounds. |
| Approach: | They propose to use a more structured approach to evaluating toxicity over time . they suggest that research that relied on automatic toxicity scores may have resulted in inaccurate results. |
| Outcome: | The Perspective API has been updated to reflect the changes in toxicity scores. |
On the Role of Speech Data in Reducing Toxicity Detection Bias (2025.naacl-long)
Copied to clipboard
Samuel Bell, Mariano Coria Meglioli, Megan Richards, Eduardo Sánchez, Christophe Ropers, Skyler Wang, Adina Williams, Levent Sagun, Marta R. Costa-jussà
| Challenge: | Text toxicity detection systems produce disproportionate rates of false positives on demographic groups . toxicity classification systems often misinterpret benign group mentions as toxic . |
| Approach: | They use group annotations to compare text-based and speech-based toxicity detection systems. |
| Outcome: | The results show that access to speech data supports reduced bias against group mentions . the authors recommend improving classifiers, rather than transcription pipelines if possible . |
Hate Speech and Counter Speech Detection: Conversational Context Does Matter (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing datasets and models target hate speech but ignore context . Existing models target either hate speech or hate and counter speech but disregard context - a new study shows that context is critical to identify hate and anti-hate speech. |
| Approach: | They propose to use context to identify hate and counter speech in a reddit conversation thread. |
| Outcome: | The proposed model improves when and why context is taken into account. |
Humans Need Context, What about Machines? Investigating Conversational Context in Abusive Language Detection (2024.lrec-main)
Copied to clipboard
| Challenge: | In this paper, we examine the role of conversational context in abusive language detection . prior studies have ignored the contextual nature of abusive language, ignoring this aspect . toxicity, hate speech, harmful stereotypes are among the forms of harmful language . |
| Approach: | They propose to use conversational context to analyze abusive language detection using two methods . they use "abusive language" as an umbrella term to refer to various forms of harmful language . |
| Outcome: | The proposed approach is based on two datasets in English and a new dataset of French tweets annotated for hate speech and stereotypes. |
Detecting Community Sensitive Norm Violations in Online Conversations (2021.findings-emnlp)
Copied to clipboard
Chan Young Park, Julia Mendelsohn, Karthik Radhakrishnan, Kinjal Jain, Tushar Kanakagiri, David Jurgens, Yulia Tsvetkov
| Challenge: | Existing efforts to identify unacceptable behavior have focused on toxicity as the sole form of community norm violation. |
| Approach: | They propose a dataset that focuses on a more complete spectrum of community norms and their violations in local conversational and global contexts. |
| Outcome: | The proposed model improves the detection of community norm violations in local conversational and global contexts. |
ModelCitizens: Representing Community Voices in Online Safety (2025.emnlp-main)
Copied to clipboard
Ashima Suvarna, Christina A Chance, Karolina Naranjo, Hamid Palangi, Sophie Hao, Thomas Hartvigsen, Saadia Gabriel
| Challenge: | Existing toxic language detection models are trained on annotations that collapse diverse perspectives into a single ground truth. |
| Approach: | They propose to augment social media posts with conversational scenarios to reflect the impact of conversational context on toxicity. |
| Outcome: | The proposed model outperforms existing models on social media with conversational scenarios. |
Mitigating Toxic Degeneration with Empathetic Data: Exploring the Relationship Between Toxicity and Empathy (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent work on controllable text generation has shown promise in successfully altering such text attributes. |
| Approach: | They propose to use empathetic data to reduce the toxicity of generated text by strategically sampling data based on empathy scores. |
| Outcome: | The proposed model significantly reduces the size of fine-tuning data to 7.5-30k samples while making significant improvements over state-of-the-art toxicity mitigation. |
The Relevance of Value Systems for Offensive Language Detection (2026.eacl-long)
Copied to clipboard
| Challenge: | Recent research in perspectivism has departed from the assumption that offensiveness can be defined through a universal perspective. |
| Approach: | They propose to use a dataset consisting of neutrally-phrased sentences on controversial topics, evaluated by individuals from 4 different value systems to identify offensiveness patterns. |
| Outcome: | The proposed dataset consists of neutrally-phrased sentences on controversial topics, evaluated by individuals from 4 different value systems. |
Mind Your Bias: A Critical Review of Bias Detection Methods for Contextual Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for detection of biases in contextual language models are inconsistent and inconclusive. |
| Approach: | They propose to use word embedding association test to detect biases in contextual language models to compare them with other methods. |
| Outcome: | The proposed methods are inconsistent and inconclusive for language models with word embeddings. |
Challenges in Automated Debiasing for Toxic Language Detection (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for debiasing toxic language data are limited in their ability to prevent biased behavior in toxic language detection systems. |
| Approach: | They propose to debiase toxic language detection models using lexical and dialectal markers using synthetic labels instead of traditional methods. |
| Outcome: | The proposed method reduces dialectal associations with toxicity despite the use of synthetic labels . |