Sheep’s Skin, Wolf’s Deeds: Are LLMs Ready for Metaphorical Implicit Hate Speech? (2025.acl-long)
Copied to clipboard
| Challenge: | specialized models fail to detect implicit hate speech due to its indirectly expressed hateful intent . advanced LLMs often misinterpret metaphorical implicit hate content, resulting in its propagation . |
| Approach: | They propose a Jailbreaking strategy and Energy-based Constrained Decoding techniques to detect implicit hate speech in large language models. |
| Outcome: | The proposed model can generate metaphorical implicit hate speech, but it fails to detect it effectively. |
Similar Papers
Probing LLMs for hate speech detection: strengths and vulnerabilities (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent efforts to detect hateful or toxic language using large language models have not used explanation, additional context and victim community information in the detection process. |
| Approach: | They use different prompt variations, input information and victim community information to evaluate large language models in zero shot setting without adding any in-context examples. |
| Outcome: | The proposed models perform significantly better when included in the pipeline than baseline models. |
Decoding Hate: Exploring Language Models’ Reactions to Hate Speech (2025.naacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are trained on vast amounts of unmoderated internet data, enabling them to generate text autonomously. |
| Approach: | They investigate the responses of seven state-of-the-art Large Language Models (LLMs) to hate speech by qualitative analysis. |
| Outcome: | The proposed models can handle hate speech inputs and mitigate it through fine-tuning and guideline guardrailing. |
Don’t Go To Extremes: Revealing the Excessive Sensitivity and Calibration Limitations of LLMs in Implicit Hate Speech Detection (2024.acl-long)
Copied to clipboard
| Challenge: | Several studies have examined whether large language models exhibit bias or discrimination against individuals or groups in terms of protected attributes like race, gender, or religion. |
| Approach: | They evaluate LLMs' ability to detect implicit hate speech and express confidence in their responses by considering prompt patterns and mainstream uncertainty estimation methods. |
| Outcome: | The proposed models exhibit two extremes: (1) excessive sensitivity towards groups or topics that may cause fairness issues, resulting in misclassifying benign statements as hate speech; (2) confidence scores for each method excessively concentrate on a fixed range, remaining unchanged regardless of the dataset’s complexity. |
Confusion is the Final Barrier: Rethinking Jailbreak Evaluation and Investigating the Real Misuse Threat of LLMs (2025.findings-emnlp)
Copied to clipboard
Yu Yan, Sheng Sun, Zhe Wang, Yijun Lin, Zenghao Duan, Zhifei Zheng, Min Liu, Zhiyi Yin, Jianping Zhang
| Challenge: | Large Language Models have been developed to deal with real-world crimes, but it remains unclear whether they internalize authentic knowledge or are forced to simulate toxic language patterns. |
| Approach: | They construct knowledge-intensive Q&A to investigate misuse threats of Large Language Models in terms of dangerous knowledge possession, harmful task planning utility, and harmfulness judgment robustness. |
| Outcome: | The findings raise concerns that jailbreak success is often attributable to a hallucination loop between jailbroken LLM and judger LLM . |
Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreaks (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods to jailbreak large language models have been poorly studied . a recent study showed that non-expert users can jailbreak LLMs by manipulating their prompts . |
| Approach: | They propose a formalism and a taxonomy of known (and possible) jailbreaks . they propose generating a dataset of model outputs across 3700 jailbreak prompts a 'prompt' attack is a new attack popularly categorized as "prompting injection attacks" |
| Outcome: | The proposed model exploits 3700 jailbreak prompts over 4 tasks to analyze their effectiveness . authors show that the model can learn to perform a new task on unseen examples . |
Confident, Calibrated, or Complicit: Safety Alignment and Ideological Bias in LLM Hate Speech Detection (2026.acl-long)
Copied to clipboard
| Challenge: | censored models outperform uncensoreed counterparts in accuracy and robustness, achieving 69.0% accuracy versus 64.1% strict accuracy. |
| Approach: | They examine how large language models with minimal safety alignment compare with more heavily aligned counterparts when deployed using political personas. |
| Outcome: | The proposed model outperforms uncensored models in accuracy and robustness, while uncensors are more malleable to ideological framing. |
Latent Hatred: A Benchmark for Understanding Implicit Hate Speech (2021.emnlp-main)
Copied to clipboard
Mai ElSherief, Caleb Ziems, David Muchlinski, Vaishnavi Anupindi, Jordyn Seybolt, Munmun De Choudhury, Diyi Yang
| Challenge: | Existing studies on explicit or overt hate speech have failed to address a more pervasive form based on coded or indirect language. |
| Approach: | They propose a theoretically-justified taxonomy of implicit hate speech and a benchmark corpus with fine-grained labels for each message and its implication. |
| Outcome: | The proposed dataset will serve as a useful benchmark for understanding this multifaceted issue. |
from Benign import Toxic: Jailbreaking the Language Model via Adversarial Metaphors (2025.acl-long)
Copied to clipboard
| Challenge: | Recent studies have exposed the risk of Large Language Models (LLMs) generating harmful content by jailbreak attacks. |
| Approach: | They propose a framework that exploits AdVersArial meTAphoR to induce LLMs to calibrate harmful metaphors for jailbreaking. |
| Outcome: | The proposed framework can successfully jailbreak Large Language Models (LLMs) by leveraging the AdVersArial meTAphoR (AVATAR) framework achieves state-of-the-art attack success rate across multiple advanced LLMs. |
HARE: Explainable Hate Speech Detection with Step-by-Step Reasoning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent benchmarks have attempted to identify and explain hate speech but lack the reasoning to supervise detection models. |
| Approach: | They propose a framework that uses large language models to fill in the gaps in hate speech explanations by using existing annotations. |
| Outcome: | The proposed framework outperforms baselines on SBIC and Implicit Hate using model-generated data and improves generalization to unseen datasets. |
How to Protect Yourself from 5G Radiation? Investigating LLM Responses to Implicit Misinformation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Current studies evaluate LLMs on explicit false statements, overlooking how misinformation manifests subtly as unchallenged premises in real-world interactions. |
| Approach: | They propose to use EchoMist to analyze implicit misinformation from diverse sources . they also investigate two mitigation methods to enhance LLMs’ capability to counter implicit mis information. |
| Outcome: | The proposed model fails to detect false premises and generate counterfactual explanations. |