PluRule: A Benchmark for Moderating Pluralistic Communities on Social Media (2026.acl-long)
Copied to clipboard
| Challenge: | Social media are shifting towards community-governed platforms where groups define their own norms. |
| Approach: | They propose a multimodal, multilingual benchmark for detecting 13,371 rule violations across 1,989 Reddit communities . they show that bigger models and increased context provide marginal gains, and universal rules like civility and self-promotion are easier to detect. |
| Outcome: | The proposed model can detect 13,371 rule violations across 1,989 Reddit communities across 2,885 rules in 9 languages. |
Similar Papers
Detecting Community Sensitive Norm Violations in Online Conversations (2021.findings-emnlp)
Copied to clipboard
Chan Young Park, Julia Mendelsohn, Karthik Radhakrishnan, Kinjal Jain, Tushar Kanakagiri, David Jurgens, Yulia Tsvetkov
| Challenge: | Existing efforts to identify unacceptable behavior have focused on toxicity as the sole form of community norm violation. |
| Approach: | They propose a dataset that focuses on a more complete spectrum of community norms and their violations in local conversational and global contexts. |
| Outcome: | The proposed model improves the detection of community norm violations in local conversational and global contexts. |
Multilingual Content Moderation: A Case Study on Reddit (2023.eacl-main)
Copied to clipboard
| Challenge: | a growing need for AI moderators to safeguard users and protect mental health of human moderator from traumatic content. |
| Approach: | They propose to use a multilingual dataset to study the challenges of content moderation . they propose to analyze 1.8 million Reddit comments in English, german, spanish and french . |
| Outcome: | The proposed dataset highlights the challenges and suggests related research problems . it shows that the proposed model can be used to predict the violated rule . |
MoMoE: Mixture of Moderation Experts Framework for AI-Assisted Online Governance (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches for content moderation require a separate model for every community and are opaque in their decision-making. |
| Approach: | They propose a modular framework that adds post-hoc explanations to enable scalable content moderation. |
| Outcome: | The proposed framework yields scalable, transparent moderation without fine-tuning across domains. |
SuperTweetEval: A Challenging, Unified and Heterogeneous Benchmark for Social Media NLP Research (2023.findings-emnlp)
Copied to clipboard
Dimosthenis Antypas, Asahi Ushio, Francesco Barbieri, Leonardo Neves, Kiamehr Rezaee, Luis Espinosa-Anke, Jiaxin Pei, Jose Camacho-Collados
| Challenge: | specialised language models (LMs) have shown to exhibit lower perplexity and higher downstream performance across the board. |
| Approach: | They propose a benchmark for NLP evaluation in social media, SuperTweetEval. |
| Outcome: | The proposed benchmark shows that social media models perform better when compared to general-purpose models, metrics and benchmarks. |
ModelCitizens: Representing Community Voices in Online Safety (2025.emnlp-main)
Copied to clipboard
Ashima Suvarna, Christina A Chance, Karolina Naranjo, Hamid Palangi, Sophie Hao, Thomas Hartvigsen, Saadia Gabriel
| Challenge: | Existing toxic language detection models are trained on annotations that collapse diverse perspectives into a single ground truth. |
| Approach: | They propose to augment social media posts with conversational scenarios to reflect the impact of conversational context on toxicity. |
| Outcome: | The proposed model outperforms existing models on social media with conversational scenarios. |
MULTIGUARD: An Efficient Approach for AI Safety Moderation Across Languages and Modalities (2025.emnlp-main)
Copied to clipboard
Sahil Verma, Keegan Hines, Jeff Bilmes, Charlotte Siska, Luke Zettlemoyer, Hila Gonen, Chandan Singh
| Challenge: | Existing approaches to detect harmful queries to large language models are fallible and vulnerable to attacks that exploit mismatched generalization of model capabilities. |
| Approach: | They propose an approach to detect harmful queries to large language models (LLMs) OMNIGUARD identifies internal representations of an LLM/MLLM that are aligned across languages or modalities and builds a language-agnostic or modality-adic classifier for detecting harmful prompts. |
| Outcome: | OMNIGUARD improves harmful prompt classification accuracy by 11.57% over the strongest baseline in a multilingual setting, by 20.44% for image-based prompts, and sets a new SOTA for audio-based ones. |
Using RL to Identify Divisive Perspectives Improves LLMs Abilities to Identify Communities on Social Media (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Experimental results show improvements on Reddit and Twitter data . |
| Approach: | They propose to take advantage of Large Language Models (LLMs) to better identify user communities. |
| Outcome: | The proposed model improves on Reddit and Twitter data and tasks of community detection, bot detection, and news media profiling. |
Watching the AI Watchdogs: A Fairness and Robustness Analysis of AI Safety Moderation Classifiers (2025.naacl-short)
Copied to clipboard
| Challenge: | ASM classifiers are designed to moderate content on social media platforms and serve as guardrails that prevent Large Language Models (LLMs) from being fine-tuned on unsafe inputs. |
| Approach: | They examine the fairness and robustness of four widely-used, closed-source ASM classifiers: OpenAI Moderation API, Perspective API, Google Cloud Natural Language (GCNL) API, and Clarifai API. |
| Outcome: | The classifiers do not unfairly classify content belonging to minority groups as unsafe compared to those belonging to majority groups and their behavior remains robust and consistent across similar inputs. |
Toxicity Detection is NOT all you Need: Measuring the Gaps to Supporting Volunteer Content Moderators through a User-Centric Method (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing efforts to automate content moderation have focused on identifying toxic, offensive, and hateful content . yet, it remains unclear whether improvements have addressed the needs of volunteer content moderators . |
| Approach: | They propose to use a model review to examine the availability of moderators' models to flag violations of various forum rules. |
| Outcome: | The proposed models perform poorly on a significant portion of the rules. |
SLM-Mod: Small Language Models Surpass LLMs at Content Moderation (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are expensive to query in real-time and do not allow for a community-specific approach to content moderation. |
| Approach: | They propose to use small language models for community-specific content moderation tasks by fine-tuning and evaluating their performance against larger open- and closed-sourced models. |
| Outcome: | The proposed models outperform zero-shot LLMs in content moderation tasks with 11.5% higher accuracy and 25.7% higher recall across all communities. |