Papers by Daniel Fried

Modular Networks for Compositional Instruction Following (2021.naacl-main)

Challenge:	Existing approaches to question generation for interactive retrieval have constrained answer spaces, limiting the amount of information a model can gain in a single turn.
Approach:	They propose a method that incorporates presupposition handling into question selection and belief updates.
Outcome:	The proposed method increases accuracy over the past state-of-the-art by 14% while resulting in 48% more efficient games in human evaluations.

Copied to clipboard

Rodolfo Corona, Daniel Fried, Coline Devin, Dan Klein, Trevor Darrell

Challenge:	Standard instruction following models struggle on novel compositions of subgoals observed during training.
Approach:	They propose a modular architecture that follows natural language instructions that describe sequences of diverse subgoals.
Outcome:	The proposed architecture improves generalization to novel subgoals and environments unseen in training.

ECCO: Can We Improve Model-Generated Code Efficiency Without Sacrificing Functional Correctness? (2024.emnlp-main)

Copied to clipboard

Siddhant Waghjale, Vishruth Veerendranath, Zhiruo Wang, Daniel Fried

Pragmatic Inference with a CLIP Listener for Contrastive Captioning (2023.findings-acl)

Challenge:	Current methods for optimizing program efficiency improve performance measured by execution time, but they often come at the cost of severely decreasing the functional correctness.
Approach:	They propose a reproducible benchmark for evaluating program efficiency via two paradigms: natural language (NL) based code generation and history-based code editing.
Outcome:	The proposed approach improves performance while maintaining correctness while adding execution information.

Copied to clipboard

Jiefu Ou, Benno Krojer, Daniel Fried

Reference-Centric Models for Grounded Collaborative Dialogue (2021.emnlp-main)

Challenge:	a new method for contrastive captioning generates discriminative captions that distinguish target images from very similar alternative distractor images.
Approach:	They propose a pragmatic inference procedure that formulates captioning as a reference game between a speaker and a listener.
Outcome:	The proposed method outperforms previous methods for discriminative captioning by 11% to 15% accuracy in human evaluations.

Copied to clipboard

Daniel Fried, Justin Chiu, Dan Klein

Dialogue Systems & Conversational Agents

Challenge:	Using a structured referent grounding module, we can effectively ground and inform a partner's utterances to their own context.
Approach:	They propose a grounded neural dialogue model that works with people in a partially-observable reference game.
Outcome:	The proposed model outperforms state-of-the-art models on a spatial grounding dialogue task and achieves a 20% relative improvement in human evaluations.

G3: Geolocation via Guidebook Grounding (2022.findings-emnlp)

Copied to clipboard

Grace Luo, Giscard Biamby, Trevor Darrell, Daniel Fried, Anna Rohrbach

Evaluating Large Language Model Biases in Persona-Steered Generation (2024.findings-acl)

Challenge:	a new task uses explicit knowledge from human-written guidebooks to improve geolocation accuracy . a state-of-the-art image-only method is unable to predict the location of an image .
Approach:	They propose a task that uses streetview images and a guidebook to predict a country for each image . they add clues from the guidebook and supervise attention with country-level pseudo labels .
Outcome:	The proposed method outperforms state-of-the-art image-only geolocation methods with 5% improvement in Top-1 accuracy.

Copied to clipboard

Andy Liu, Mona Diab, Daniel Fried

Identifying & Interactively Refining Ambiguous User Goals for Data Visualization Code Generation (2025.emnlp-main)

Challenge:	a recent wave of powerful new large language models has raised concerns that their expressed opinions may be biased towards certain political, national or moral viewpoints.
Approach:	They define an incongruous persona as a persona with multiple traits where one trait makes its other traits less likely in human survey data.
Outcome:	The results show that LLMs are less steerable towards incongruous personas than congruous ones . the models that are fine-tuned with RLHF are more steerable, especially towards stances associated with political liberals and women .

Copied to clipboard

Mert Inan, Anthony Sicilia, Alex Xie, Saujas Vaduguru, Daniel Fried, Malihe Alikhani

Inferring Rewards from Language in Context (2022.acl-long)

Challenge:	ambiguities in natural language can lead to outputs that seem correct but fail to reflect the speaker’s intent.
Approach:	They propose to identify and then resolve ambiguities in natural language and propose metrics to quantify them.
Outcome:	The proposed metrics better correlate with human annotations than uncertainty baselines.

Copied to clipboard

Jessy Lin, Daniel Fried, Dan Klein, Anca Dragan

Challenge:	a new study grounding language to reward functions extends the standard instruction following setup in this way.
Approach:	They propose a model that infers rewards from language pragmatically by reasoning about how speakers choose utterances to elicit desired actions and reveal information about their preferences.
Outcome:	The proposed model infers rewards from language pragmatically on a flight–booking task with natural language.

Success and Cost Elicit Convention Formation for Efficient Communication (2026.acl-long)

Copied to clipboard

Saujas Vaduguru, Yilun Hua, Yoav Artzi, Daniel Fried

Dialogue Systems & Conversational Agents

Challenge:	ad hoc conventions allow people to coordinate on short, less costly utterances that are understood using shared conversational context.
Approach:	They propose a method to train large multimodal models to form conventions . they use simulated reference games to produce training data .
Outcome:	The proposed method reduces message length by up to 41% while increasing success by 15% over the course of the interaction.

Is the Pope Catholic? Yes, the Pope is Catholic. Generative Evaluation of Non-Literal Intent Resolution in LLMs (2024.acl-short)

Copied to clipboard

Akhila Yerukola, Saujas Vaduguru, Daniel Fried, Maarten Sap

Challenge:	Existing work on discriminative evaluations of large language models has focused on discrimination, but this paper examines their intention understanding by examining their responses to non-literal utterances.
Approach:	They propose a framework to evaluate large language models’ intention understanding by examining their responses to non-literal utterances.
Outcome:	The proposed framework compares large language models' responses to human-like expectations and provides nuanced evaluations of their intention understanding.

Are You Looking? Grounding to Multiple Modalities in Vision-and-Language Navigation (P19-1)

Copied to clipboard

Ronghang Hu, Daniel Fried, Anna Rohrbach, Dan Klein, Trevor Darrell, Kate Saenko

Contrastive Decoding: Open-ended Text Generation as Optimization (2023.acl-long)

Challenge:	Existing models that ground language into visual appearance and route structure are outperforming their visual counterparts in unseen new environments.
Approach:	They propose to decompose the grounding procedure into a set of expert models with access to different modalities and ensemble them at prediction time.
Outcome:	The proposed model outperforms models with only route structure and visual features on the benchmark Room-to-Room dataset.

Copied to clipboard

Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori Hashimoto, Luke Zettlemoyer, Mike Lewis

Language Models

Challenge:	Using a language model, maximum probability is a poor decoding objective because it produces short and repetitive text.
Approach:	They propose a reliable decoding approach that optimizes a contrastive objective subject to a plausibility constraint.
Outcome:	The proposed approach outperforms four strong decoding algorithms in automatic and human evaluations across wikipedia, news and story domains.

mrCAD: Multimodal Communication to Refine Computer-aided Designs (2025.findings-emnlp)

Copied to clipboard

William P McCarthy, Saujas Vaduguru, Karl D.d. Willis, Justin Matejka, Judith E Fan, Daniel Fried, Yewen Pu

Execution-Based Evaluation for Open-Domain Code Generation (2023.findings-emnlp)

Challenge:	generative AI excels at creating artifacts in a single turn but can struggle to make precise refinements that match our design intent.
Approach:	They propose to use multi-turn interactions to iterate and refine computer-aided designs (CADs) they use text and drawing to communicate with each other over multiple rounds of interaction .
Outcome:	mrCAD consists of 6,082 communication games, 15,163 instruction-execution rounds, played between 1,092 pairs of humans.

Copied to clipboard

Zhiruo Wang, Shuyan Zhou, Daniel Fried, Graham Neubig

Natural Language to Code Translation with Execution (2022.emnlp-main)

Challenge:	ODEX is the first open-domain EXecution-based natural language (NL) to Python code generation dataset.
Approach:	They propose to use a dataset to extend the scope of coding queries to more realistic settings by using open-domain EXecution-based natural language (NL) to Python.
Outcome:	The proposed dataset has 945 NL-Code pairs and 1,707 human-written test cases.

Copied to clipboard

Freda Shi, Daniel Fried, Marjan Ghazvininejad, Luke Zettlemoyer, Sida I. Wang

Text Generation

Challenge:	Generative code models do not explicitly incorporate program semantics during training, but they are able to generate correct solutions for many problems.
Approach:	They introduce execution result-based minimum Bayes risk decoding for program selection . they select output programs from a generated candidate set by marginalizing over implementations that share the same semantics .
Outcome:	The proposed model outperforms all other methods on natural language-to-code translation.

API-Assisted Code Generation for Question Answering on Varied Table Structures (2023.emnlp-main)

Copied to clipboard

Yihan Cao, Shuyi Chen, Ryan Liu, Zhiruo Wang, Daniel Fried

Improving Model Factuality with Fine-grained Critique-based Evaluator (2025.acl-long)

Challenge:	Existing approaches to table question answering have been limited to specific table structures.
Approach:	They propose a unified TableQA framework that uses Python as a querying language and few-shot prompting to translate NL questions into Python programs.
Outcome:	The proposed framework provides a unified representation for structured tables as multi-index Pandas data frames and uses Python as a powerful querying language to translate NL questions into Python programs.

Copied to clipboard

Yiqing Xie, Wenxuan Zhou, Pradyot Prakash, Di Jin, Yuning Mao, Quintin Fettes, Arya Talebzadeh, Sinong Wang, Han Fang, Carolyn Rose, Daniel Fried, Hejia Zhang

Responsible & Trustworthy NLP

Challenge:	Factuality evaluation aims to detect factual errors produced by language models and guide the development of more factual models.
Approach:	They propose a framework that leverages FenCE to improve the factuality of LM generators by constructing training data.
Outcome:	The proposed framework improves the factuality of LM generators by enhancing their training data.

Syntactic Structure Distillation Pretraining for Bidirectional Encoders (2020.tacl-1)

Copied to clipboard

Adhiguna Kuncoro, Lingpeng Kong, Daniel Fried, Dani Yogatama, Laura Rimell, Chris Dyer, Phil Blunsom

Pragmatics in Language Grounding: Phenomena, Tasks, and Modeling Approaches (2023.findings-emnlp)

Challenge:	Textual representation learners trained on large amounts of data have been successful on downstream tasks.
Approach:	They propose a knowledge distillation strategy for injecting syntactic biases into BERT pretraining by distilling the approximate marginal distribution over words in context from the syntaktic LM.
Outcome:	The proposed method reduces relative error by 2–21% on a diverse set of structured prediction tasks.

Copied to clipboard

Daniel Fried, Nicholas Tomlin, Jennifer Hu, Roma Patel, Aida Nematzadeh

VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks (2024.acl-long)

Challenge:	People rely heavily on context to enrich meaning beyond what is literally said.
Approach:	They analyze how task goals, environmental contexts, and communicative affordances in each work enrich linguistic meaning.
Outcome:	The proposed frameworks are based on linguistic goals, environmental contexts, and communicative affordances to enrich linguistic meaning.

Copied to clipboard

Jing Yu Koh, Robert Lo, Lawrence Jang, Vikram Duvvur, Ming Lim, Po-Yu Huang, Graham Neubig, Shuyan Zhou, Russ Salakhutdinov, Daniel Fried

Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs (2022.emnlp-main)

Challenge:	Existing benchmarks focus on text-based agents, neglecting many natural tasks that require visual information to effectively solve.
Approach:	They propose a benchmark to assess the performance of multimodal web agents . they use visual and textual inputs to process and interpret natural language instructions .
Outcome:	a new benchmark assesses the performance of multimodal agents on visually grounded tasks . the benchmark identifies limitations of text-only agents and offers insights towards building stronger agents for the web .

Copied to clipboard

Maarten Sap, Ronan Le Bras, Daniel Fried, Yejin Choi

AutoReply: Detecting Nonsense in Dialogue with Discriminative Replies (2023.findings-emnlp)

Challenge:	We show that one of today’s largest language models lacks this kind of social intelligence out-of-the-box, using two tasks: SocialIQa and ToMi.
Approach:	They propose to use social intelligence and Theory of Mind to examine whether modern large-scale language models lack this kind of social intelligence out-of-the-box.
Outcome:	The proposed model lacks social intelligence out-of-the-box, and has well-below human accuracies on SocialIQa and ToMi, respectively.

Copied to clipboard

Weiyan Shi, Emily Dinan, Adi Renduchintala, Daniel Fried, Athul Jacob, Zhou Yu, Mike Lewis

Dialogue Systems & Conversational Agents

Challenge:	Existing models for dialogue detection make many errors in their own messages . a dataset of long dialogues richly grounded in the game state contains many errors .
Approach:	They propose to use an annotated dialogue dataset to generate automatic responses for dialogue models.
Outcome:	The proposed model outperforms handcrafted replies and performs on par with supervised learning approaches.

CRScore: Grounding Automated Evaluation of Code Review Comments in Code Claims and Smells (2025.naacl-long)

Copied to clipboard

Atharva Naik, Marcus Alenius, Daniel Fried, Carolyn Rose

CodeRAG-Bench: Can Retrieval Augment Code Generation? (2025.findings-naacl)

Challenge:	Current review comment evaluation metrics rely on comparisons with a human-written reference for a given code change (also called a diff).
Approach:	They propose to use a reference-free metric to measure review quality like conciseness, comprehensiveness, and relevance to compare the quality of code changes with human-written references.
Outcome:	The proposed metric can produce fine-grained scores that have the greatest alignment with human judgment and are more sensitive than reference-based metrics.

Copied to clipboard

Zora Zhiruo Wang, Akari Asai, Xinyan Velocity Yu, Frank F. Xu, Yiqing Xie, Graham Neubig, Daniel Fried

Pragmatically Informative Text Generation (N19-1)

Challenge:	Language models excel at generating code, but many programs are difficult to generate using only parametric knowledge.
Approach:	They propose a retrieval-augmented code generation benchmark that provides reproducible evaluations on retrieval and end-to-end code generation performance.
Outcome:	The proposed benchmark covers programming, open-domain, and repository-level tasks and provides reproducible evaluations on retrieval and end-to-end code generation performance.

Copied to clipboard

Sheng Shen, Daniel Fried, Jacob Andreas, Dan Klein

Rewarding the Unlikely: Lifting GRPO Beyond Distribution Sharpening (2025.emnlp-main)

Challenge:	Existing approaches to pragmatics have been used to improve the informativeness of generated text in grounded language learning problems.
Approach:	They propose to use pragmatics to improve the informativeness of conditional text models . they propose to apply pragmatic reasoning to more traditional language generation tasks .
Outcome:	The proposed methods improve the performance of strong existing systems for abstractive summarization and generation from structured meaning representations.

Copied to clipboard

Andre Wang He, Daniel Fried, Sean Welleck

Language Models

Challenge:	Reinforcement learning is emerging as a primary driver for improving language model reasoning capabilities.
Approach:	They propose a method for explicitly up-weighting rare but correct solutions to overcome rank bias in group relative policy optimization (GRPO) .
Outcome:	The proposed method mitigates rank bias and improves pass@N across a large range of N in both synthetic and real theorem proving settings.

Unified Pragmatic Models for Generating and Following Instructions (N18-1)

Copied to clipboard

Daniel Fried, Jacob Andreas, Dan Klein

Explainability & Interpretability in NLP

Challenge:	a new technique for layering explicit pragmatic inference on top of models for sequential tasks is proposed . explicit pragmatic reasoning is used to generate and follow natural language instructions .
Approach:	They propose a pragmatic speaker that uses the base listener to simulate the interpretation of candidate descriptions and a listener that reasons counterfactually about alternative descriptions.
Outcome:	The proposed model improves state-of-the-art models for interpreting human instructions and speaker models in diverse settings.

Cross-Domain Generalization of Neural Constituency Parsers (P19-1)

Copied to clipboard

Daniel Fried, Nikita Kitaev, Dan Klein

Syntactic Text Processing

Challenge:	Neural parsers perform well on in-domain benchmarks, but their performance degrades in well-understood ways.
Approach:	They analyze generalization on English and Chinese corpora to see if they can generalize to other domains.
Outcome:	The proposed neural parsers perform better on in-domain benchmarks than on out-of-domain corpora.

Data Augmentation for Code Translation with Comparable Corpora and Multiple References (2023.findings-emnlp)

Copied to clipboard

Yiqing Xie, Atharva Naik, Daniel Fried, Carolyn Rose

Text Generation

Challenge:	Existing methods for translating code between programming languages are limited by parallel training data.
Approach:	They propose a data augmentation technique that builds comparable corpora and augments existing parallel data with multiple reference translations.
Outcome:	The proposed techniques improve CodeT5 translation between Java, Python, and C++ by an average of 7.5% Computational Accuracy (CA@1) .

Symbolic Planning and Code Generation for Grounded Dialogue (2023.emnlp-main)

Copied to clipboard

Justin Chiu, Wenting Zhao, Derek Chen, Saujas Vaduguru, Alexander Rush, Daniel Fried