Papers by Stephen Bach
Preference Tuning For Toxicity Mitigation Generalizes Across Languages (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Detoxifying multilingual Large Language Models (LLMs) has become crucial due to their increasing global use. |
| Approach: | They propose to use English preference tuning to study cross-lingual detoxification of LLMs. |
| Outcome: | The proposed method reduces toxicity in multilingual LLMs by reducing the probability of mGPT-1.3B generating toxic continuations across 17 languages. |
The State of Multilingual LLM Safety Research: From Measuring The Language Gap To Mitigating It (2025.emnlp-main)
Copied to clipboard
| Challenge: | a systematic review of 300 publications reveals a language gap in LLM safety research . even high-resource non-English languages receive little attention, authors note . |
| Approach: | They propose to focus on safety evaluation, training data generation, and crosslingual safety generalization based on their findings. |
| Outcome: | The authors suggest that the field can develop more robust, inclusive safety practices for diverse global populations. |
PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts (2022.acl-demo)
Copied to clipboard
Stephen Bach, Victor Sanh, Zheng Xin Yong, Albert Webson, Colin Raffel, Nihal V. Nayak, Abheesht Sharma, Taewoon Kim, M Saiful Bari, Thibault Fevry, Zaid Alyafeai, Manan Dey, Andrea Santilli, Zhiqing Sun, Srulik Ben-david, Canwen Xu, Gunjan Chhablani, Han Wang, Jason Fries, Maged Al-shaibani, Shanya Sharma, Urmish Thakker, Khalid Almubarak, Xiangru Tang, Dragomir Radev, Mike Tian-jian Jiang, Alexander Rush
| Challenge: | PromptSource is a system for creating, sharing, and using natural language prompts . prompts are used to train and query language models in zero-shot learning settings . |
| Approach: | PromptSource is a system for creating, sharing, and using natural language prompts . et al.: using prompts to train and query language models is emerging area in NLP . they propose a templating language for defining data-linked prompts, a user interface that iterates on prompt development . |
| Outcome: | PromptSource is a system for creating, sharing, and using natural language prompts . it has a templating language for defining data-linked prompts and a community-driven set of guidelines . |
If CLIP Could Talk: Understanding Vision-Language Model Representations Through Their Preferred Concept Descriptions (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies assume that VLMs prioritize visual attributes to represent concepts. |
| Approach: | They propose a novel approach to characterize features important for VLMs using reinforcement learning. |
| Outcome: | The proposed approach characterizes features that are important for VLMs . it shows that spurious descriptions have a major role in VLM representations despite providing no helpful information. |
Trove: A Flexible Toolkit for Dense Retrieval (2026.eacl-demo)
Copied to clipboard
| Challenge: | Existing retrieval tools require considerable engineering effort for many tasks like efficient data management or model customization. |
| Approach: | They propose a novel open-source retrieval toolkit that simplifies research experiments without sacrificing flexibility or speed. |
| Outcome: | The proposed tool reduces memory consumption by 2.6 and allows for arbitrary customizations. |
Revisiting Generalization Across Difficulty Levels: It’s Not So Easy (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing research is mixed regarding whether training on easier or harder data leads to better results. |
| Approach: | They examine how well large language models generalize across different task difficulties by using a large dataset and a well-established difficulty metric. |
| Outcome: | The results show that training on hard data can't achieve consistent improvements across the full range of difficulties. |
Alfred: A System for Prompted Weak Supervision (2023.acl-demo)
Copied to clipboard
| Challenge: | Alfred is the first system for programmatic weak supervision (PWS) that creates training data for machine learning by prompting. |
| Approach: | They propose to use Python to create training data by prompting for machine learning . they find that it improves query throughput by 2.9x versus a naive approach . |
| Outcome: | The proposed system improves query throughput by 2.9x versus a naive approach. |
Beyond Contrastive Learning: Synthetic Data Enables List-wise Training with Multiple Levels of Relevance (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new approach to training with binary relevance labels uses synthetic data . contrastive learning with binary correlations leaves out subtle nuances useful for ranking . |
| Approach: | They propose to use waterstein distance as a loss function for training transformer-based retrievers with graduated relevance labels instead of real documents. |
| Outcome: | The proposed method outperforms conventional training with InfoNCE by a large margin on MARCO and BEIR benchmarks without using real documents. |
Planetarium: A Rigorous Benchmark for Translating Text to Structured Planning Languages (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing evaluation methods struggle to ensure semantic correctness and rely on simple or unrealistic datasets. |
| Approach: | They propose a benchmark to evaluate language models’ ability to generate PDDL code from natural language descriptions of planning tasks. |
| Outcome: | The proposed benchmark evaluates the ability of language models to generate PDDL code from natural language descriptions of planning tasks against ground truth and a dataset of 145,918 text-to-PDDL pairs with varying levels of difficulty. |
Learning to Generate Instruction Tuning Datasets for Zero-Shot Task Adaptation (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing instruction tuning datasets are limited by the quality of the instruction tuning data. |
| Approach: | They propose a model that converts unannotated text into task-specific training datasets for instruction tuning. |
| Outcome: | The proposed model improves the performance of pretrained and instruction tuned models over the de facto self-supervised baseline. |
LexC-Gen: Generating Data for Extremely Low-Resource Languages with Large Language Models and Bilingual Lexicons (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing word-to-word translations from labeled task data in low-resource languages have limited lexical overlap with task data. |
| Approach: | They propose a method that generates low-resource-language classification task data at scale using bilingual lexicons. |
| Outcome: | The proposed method improves on 17 low-resource languages with bilingual lexicons compared with existing models on sentiment analysis and topic classification tasks. |
Does CLIP Bind Concepts? Probing Compositionality in Large Image Models (2024.findings-eacl)
Copied to clipboard
| Challenge: | Large-scale neural network models combining text and images have made incredible progress in recent years, but to what extent they encode compositional representations of the concepts over which they operate remains an open question . |
| Approach: | They compare the performance of a large pretrained vision and language model (CLIP) to a set of three synthetic datasets designed to test concept binding. |
| Outcome: | The proposed model can encode compositional concepts and bind variables in a structure-sensitive way, e.g., differentiating ‘cube behind sphere’ from ‘cub behind cube’. |