Papers by Dawn Song
Imitation Attacks and Defenses for Black-box Machine Translation Systems (2020.emnlp-main)
Copied to clipboard
| Challenge: | Using simulated experiments, we demonstrate that MT systems can be stolen even when imitation models have different input data or architectures than their target models. |
| Approach: | They propose a defense that modifies translation outputs to misdirect optimization of imitation models. |
| Outcome: | The proposed defense degrades the adversary’s BLEU score and attack success rate at some cost in the defender’s performance and inference speed. |
Fico: Evaluating Vision-Language Models under Visual Fidelity and Compression at Scale (2026.findings-acl)
Copied to clipboard
| Challenge: | Visual text compression is emerging paradigm for rendering text as images for processing by vision-language models. |
| Approach: | They propose a benchmark to assess VLM robustness under dense visual inputs. |
| Outcome: | Evaluating 13 general-purpose VLMs and 3 OCR-specialized models reveals performance drops sharply under increased density or reduced resolution; cross-task transfer between OCR, NIAH, and VQA is limited; and VQ is comparatively robust because low-level details are lost before high-level semantics. |
Zero-Shot Information Extraction as a Unified Text-to-Triple Translation (2021.emnlp-main)
Copied to clipboard
| Challenge: | a number of information extraction tasks require task-specific training. |
| Approach: | They propose a text-to-triple translation framework for information extraction tasks . they propose enabling task-agnostic translation by leveraging latent knowledge of a pre-trained language model . |
| Outcome: | The proposed framework outperforms the existing methods on open information extraction tasks. |
PlotCoder: Hierarchical Decoding for Synthesizing Visualization Code in Programmatic Context (2021.acl-long)
Copied to clipboard
| Challenge: | Creating effective visualizations is a challenge given the myriad of parameters that users need to provide. |
| Approach: | They propose to synthesize visualization programs from natural language utterances and programmatic context using PlotCoder. |
| Outcome: | The proposed architecture models both the code context and the input utterance. |
dLLM: Simple Diffusion Language Modeling (2026.acl-demo)
Copied to clipboard
| Challenge: | diffusion language models (DLMs) are evolving rapidly but many lack transparent implementations or are scattered across codebases. |
| Approach: | They propose an open-source framework that unifies diffusion language modeling components while remaining flexible enough to support new methods and architectures. |
| Outcome: | dLLM unifies the core components of diffusion language modeling and makes them easy to customize for new designs. |
Benchmarking Language Models for Code Syntax Understanding (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained language models capture the syntactic rules of natural languages without fine-tuning on syntax understanding tasks. |
| Approach: | They propose a benchmarking test to compare pre-trained language models with a large-scale dataset of programs annotated with syntactic relationships in their corresponding abstract syntax trees. |
| Outcome: | The proposed model fails to match baselines based on positional offsets and keywords. |
PALT: Parameter-Lite Transfer of Language Models for Knowledge Graph Completion (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained language models (LMs) are a powerful transfer learning approach for knowledge graph (KG) completion. |
| Approach: | They propose a parameter-lite transfer learning approach for pretrained language models for knowledge graph (KG) completion. |
| Outcome: | The proposed model outperforms the state-of-the-art models on a knowledge graph completion benchmark by tuning 1% of the parameters. |
AGENTVIGIL: Automatic Black-Box Red-teaming for Indirect Prompt Injection against LLM Agents (2025.findings-emnlp)
Copied to clipboard
Zhun Wang, Vincent Siu, Zhe Ye, Tianneng Shi, Yuzhou Nie, Xuandong Zhao, Chenguang Wang, Wenbo Guo, Dawn Song
| Challenge: | AGENTVIGIL is a black-box optimization framework to exploit indirect prompt injection vulnerabilities . indirect prompts compromise the core of LLM agents by manipulating contextual information rather than direct user prompts. |
| Approach: | They propose a black-box optimization framework to exploit indirect prompt injection vulnerabilities . they use a Monte Carlo tree-based algorithm to iteratively refine inputs . |
| Outcome: | The proposed framework achieves 71% and 70% success rates against two public benchmarks . |
Joint Language Semantic and Structure Embedding for Knowledge Graph Completion (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods to complete knowledge triplets rely on structures or semantics, but use semantics to improve performance. |
| Approach: | They propose to embed semantics in the natural language description of knowledge triplets with their structure information. |
| Outcome: | The proposed method improves performance on knowledge graph benchmarks and on low-resource regimes. |
IELM: An Open Information Extraction Benchmark for Pre-Trained Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies show pre-trained LMs store linguistic and relational knowledge . pre-training LM models can answer "fill-in-the-blank" questions based on pre-defined relations . |
| Approach: | They propose an open information extraction benchmark for pre-trained language models . they turn pre-trained LMs into zero-shot OIE systems to examine open relational information . |
| Outcome: | The proposed benchmark outperforms state-of-the-art methods on factual OIE datasets without training sets. |
Position: LLM Watermarking Should Align Stakeholders’ Incentives for Practical Adoption (2026.findings-acl)
Copied to clipboard
| Challenge: | despite advances in watermarking algorithms, real-world deployment remains limited . model watermarks can be used to protect intellectual property and promote trust in AI . |
| Approach: | They propose to revisit three classes of watermarking to examine incentives for large language models . model watermarks naturally aligns with LLM provider interests, yet faces new challenges in open-source ecosystems . |
| Outcome: | The proposed methods can be used in dataset decontamination, user-controlled provenance, and in-context watermarking. |
Hidden Persuaders: LLMs’ Political Leaning and Their Influence on Voters (2024.emnlp-main)
Copied to clipboard
| Challenge: | This paper examines the political leanings of large language models (LLMs) in the 2024 election. |
| Approach: | They propose to use large language models to examine users' political leanings in the 2024 presidential election to determine their political preference. |
| Outcome: | The proposed models show that they have a political leaning and can influence political views in the 2024 presidential election. |
Re-Tuning: Overcoming the Compositionality Limits of Large Language Models with Recursive Tuning (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods to solve compositional tasks are limited by complexity and complexity. |
| Approach: | They propose a method that tunes large language models to break down a problem into subproblems, solve those subproblem, and combine the results. |
| Outcome: | The proposed method significantly improves model performance on three representative compositional tasks: integer addition, dynamic programming, and parity. |
BEEAR: Embedding-based Adversarial Removal of Safety Backdoors in Instruction-tuned Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Safety backdoors in large language models can be triggered while evading detection during normal interactions. |
| Approach: | They propose a bi-level optimization method that uses a key insight: backdoor triggers induce a uniform drift in the model’s embedding space . inner level identifies universal perturbations to the decoder’s embedded spaces that steer the model towards defender-defined unwanted behaviors; outer level fine-tunes the model to reinforce safe behaviors against these perturbations. |
| Outcome: | The proposed mitigation method reduces the success rate of safety backdoor attacks from over 95% to 1% for general harmful behaviors and from 47% to 0% for Sleeper Agents, without compromising the model’s usefulness. |
COSMIC: Generalized Refusal Direction Identification in LLM Activations (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods depend on predefined refusal templates detectable in output tokens or manual review. |
| Approach: | They propose a framework that optimally identifies steering directions and target layers using cosine similarity, entirely independent of output text. |
| Outcome: | The proposed framework achieves comparable steering effectiveness without any prior knowledge or assumptions of a model’s refusal behavior such as the use of certain refusal tokens. |
SafeKey: Amplifying Aha-Moment Insights for Safety Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Reasoning Models (LRMs) introduce a new paradigm of explicitly reasoning before answering, but they pose great safety risks against harmful queries and adversarial attacks. |
| Approach: | They propose a safety aha moment that activates safety reasoning and leads to a safe response. |
| Outcome: | The proposed model can generalize to unseen jailbreak prompts while maintaining general abilities. |
Uncertainty Quantification in LLM Agents: Foundations, Emerging Challenges, and Opportunities (2026.acl-long)
Copied to clipboard
Changdae Oh, Seongheon Park, To Eun Kim, Jiatong Li, Wendi Li, Samuel Yeh, Sean Du, Hamed Hassani, Paul Bogdan, Dawn Song, Sharon Li
| Challenge: | Uncertainty quantification (UQ) for large language models is a key building block for daily applications. |
| Approach: | They propose a general formulation of agent UQ that subsumes broad classes of existing UQ setups. |
| Outcome: | The proposed framework is based on the first general formulation of agent UQ that subsumes broad classes of existing setups. |
Pretrained Transformers Improve Out-of-Distribution Robustness (2020.acl-main)
Copied to clipboard
| Challenge: | Pretrained Transformers are more effective at detecting anomalous or OOD examples, while many previous models are frequently worse than chance. |
| Approach: | They construct a new robustness benchmark with real distribution shifts to measure out-of-distribution generalization for seven NLP datasets and compare them to previous models. |
| Outcome: | The proposed model generalizations for seven datasets show that pretrained Transformers are significantly less effective at detecting anomalous or OOD examples, while many previous models are often worse than chance. |
DeepStruct: Pretraining of Language Models for Structure Prediction (2022.findings-acl)
Copied to clipboard
| Challenge: | Pretrained language models perform structural understanding tasks that focus on understanding one aspect of the text. |
| Approach: | They propose a method for improving the structural understanding abilities of language models by pretraining them to generate structures from the text on task-agnostic corpora. |
| Outcome: | The proposed model performs state-of-the-art on 21 of 28 datasets. |
Grounded Graph Decoding improves Compositional Generalization in Question Answering (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Current compositional generalization models lose syntax context when learning a flat input . a new method to improve compositional globalization is proposed to ground structured predictions with an attention mechanism. |
| Approach: | They propose a method to ground structured predictions by a structure-based attention mechanism. |
| Outcome: | The proposed method performs competitively on the Compositional Freebase Questions dataset. |