Papers by Dmitriy Bespalov
Towards Building a Robust Toxicity Predictor (2023.acl-industry)
Copied to clipboard
| Challenge: | Recent studies have focused on robustness of toxicity language predictors, but this is problematic for real-world toxicity detection. |
| Approach: | They propose a novel adversarial attack that exploits greedy search strategies to fool toxic text classifiers. |
| Outcome: | The proposed attack can detect weaker toxicity language detectors even against unseen attacks. |
TurboFuzzLLM: Turbocharging Mutation-based Fuzzing for Effectively Jailbreaking Large Language Models in Practice (2025.naacl-industry)
Copied to clipboard
| Challenge: | Existing methods for jailbreaking large-language models are limited by their limitations . authors present a mutation-based fuzzing technique that generates effective jailbreaking templates . |
| Approach: | They propose a mutation-based fuzzing technique for efficiently finding effective jailbreaking templates that combine with harmful questions to generate harmful responses. |
| Outcome: | The proposed technique achieves 95% attack success rates on public datasets for leading LLMs . it also shows impressive generalizability to unseen harmful questions and improves model defenses to prompt attacks. |
IPR: Intelligent Prompt Routing with User-Controlled Quality-Cost Trade-offs (2025.emnlp-industry)
Copied to clipboard
Aosong Feng, Balasubramaniam Srinivasan, Yun Zhou, Zhichao Xu, Kang Zhou, Sheng Guan, Yueyan Chen, Xian Wu, Ninad Kulkarni, Yi Zhang, Zhengyuan Shen, Dmitriy Bespalov, Soumya Smruti Mishra, Yifei Teng, Darren Yow-Bang Wang, Haibo Ding, Lin Lee Cheong
| Challenge: | Existing systems require users to manually select models or employ rigid routing rules that fail to capture the continuous spectrum of query complexity. |
| Approach: | They propose a quality-constrained intelligent prompt routing framework that automatically selects optimal models based on predicted response quality and user-specified tolerance levels. |
| Outcome: | The proposed framework achieves 43.9% cost reduction while maintaining quality parity with strongest model in the Claude family and processes requests with sub-150ms latency. |
TaeBench: Improving Quality of Toxic Adversarial Examples (2025.naacl-industry)
Copied to clipboard
| Challenge: | Existing adversarial examples generate invalid or ambiguous examples that fool the systems into wrong detection. |
| Approach: | They propose an annotation pipeline for quality control of generated toxic adversarial examples (TAE) they use model-based automated annotation and human-based quality verification to assess quality requirements of a TAE dataset. |
| Outcome: | The proposed pipeline can transfer-attack SOTA toxicity content moderation models and services with adversarial training. |
LaRS: Latent Reasoning Skills for Chain-of-Thought Reasoning (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods require human experts or pre-trained LLMs to describe the skill to guide the selection. |
| Approach: | They propose a new approach that uses unsupervised learning to create a latent space representation of rationales with a variable called a reasoning skill. |
| Outcome: | Empirical results show that LaRS outperforms SOTA skill-based selection methods . it processes example banks four times faster and reduces LLM inferences by half . |
Agent vs. Agent: Automated Data Generation and Red-Teaming for Custom Agentic Workflows (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existing red-teaming frameworks like AgentHarm use static prompts and hardcoded toolsets . |
| Approach: | They propose a red-teaming framework that generates adversarial tasks and evaluation functions tailored to arbitrary toolsets and uses iterative prompt refinement with self-reflection to develop more effective attacks. |
| Outcome: | The proposed approach achieves 162% increase in attack success rate on o4-mini and 86% success on gemini 2.5 Pro. |
Graph of Attacks with Pruning: Optimizing Stealthy Jailbreak Prompt Generation for Enhanced LLM Content Moderation (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existing jailbreaking methods create adversarial prompts to bypass LLM safeguards. |
| Approach: | They propose a framework for generating stealthy jailbreak prompts that enables knowledge sharing across attack paths. |
| Outcome: | The proposed framework outperforms state-of-the-art methods for attacking both open and closed LLMs with attack success rates of >96%. |