Papers by Yoav Artzi
Executing Instructions in Situated Collaborative Interactions (D19-1)
Copied to clipboard
Alane Suhr, Claudia Yan, Jack Schluger, Stanley Yu, Hadi Khader, Marwa Mouallem, Iris Zhang, Yoav Artzi
| Challenge: | a collaborative game with natural language instruction allows users to adapt to the system abilities by changing their language or deciding to accomplish tasks themselves. |
| Approach: | They propose a collaborative game where a user instructs a system to complete tasks, but acts alongside it. |
| Outcome: | The proposed game allows users to adapt to the system abilities by changing their language or deciding to accomplish tasks themselves. |
Simple Recurrent Units for Highly Parallelizable Recurrence (D18-1)
Copied to clipboard
| Challenge: | recurrent neural networks scale poorly due to the intrinsic difficulty in parallelizing their state computations. |
| Approach: | They propose a simple recurrent unit that provides expressive recurrence and allows highly parallel implementation. |
| Outcome: | The proposed model achieves 5—9x speed-up over cuDNN-optimized LSTM on classification and question answering datasets and delivers stronger results than LS and convolutional models. |
lilGym: Natural Language Visual Reasoning with Reinforcement Learning (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to language-conditioned reinforcement learning in visual environments are limited by language semantics. |
| Approach: | They propose a new benchmark for language-conditioned reinforcement learning in visual environments . they annotate 2,661 highly-compositional human-written natural language statements . |
| Outcome: | The proposed approach is based on 2,661 highly-compositional human-written natural language statements grounded in an interactive visual environment. |
Continual Learning for Grounded Instruction Generation by Observing Human Following Behavior (2021.tacl-1)
Copied to clipboard
| Challenge: | Despite its potential and prevalence, this signal is understudied for learning to generate natural language. |
| Approach: | They propose to use this signal to improve the system's ability to generate instructions via contextual bandit learning. |
| Outcome: | The proposed system improves its ability to generate natural language through interaction with users, and the results are shown. |
CoGen: Learning from Feedback with Coupled Comprehension and Generation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on coupling comprehension and generation in computational systems show that the ability to finish incomplete partner utterances in dialogue is closely related to comprehension and vice versa. |
| Approach: | They propose techniques to tightly integrate comprehension and generation capabilities with focus on continually learning from interaction with users. |
| Outcome: | The proposed models improve performance by 26% and 17% over time, while the non-coupled system is more human-like. |
Crowdsourcing Beyond Annotation: Case Studies in Benchmark Data Collection (2021.emnlp-tutorials)
Copied to clipboard
| Challenge: | Developing a theory of crowdsourcing for practical language problems remains an open challenge . |
| Approach: | This tutorial exposes NLP researchers to data collection crowdsourcing methods and principles through case studies. |
| Outcome: | This tutorial exposes NLP researchers to various data collection crowdsourcing methods and practices through case studies. |
Situated Mapping of Sequential Instructions to Actions with Single-step Reward Observation (P18-1)
Copied to clipboard
| Challenge: | Existing approaches to map context-dependent sequential instructions to actions are based on discourse and state dependencies . we evaluate on SCONE domains and show absolute accuracy improvements of 9.8%-25.3% . |
| Approach: | They propose a model that considers previous utterances and the state of the world to map sequential instructions to actions. |
| Outcome: | The proposed model improves on the SCONE domains and on the target domains. |
Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies (N18-1)
Copied to clipboard
| Challenge: | a dataset of 1.3 million articles and summaries written by authors and editors in newsrooms of 38 major news publications . identifying large, high-quality resources for summarization has called for creative solutions in the past. |
| Approach: | They present a summarization dataset of 1.3 million articles and summaries written by newsrooms of 38 major news publications. |
| Outcome: | The summarization dataset shows high diversity of summarizing styles . authors train existing methods on the data to evaluate its utility and challenges. |
Abstract Visual Reasoning with Tangram Shapes (2022.emnlp-main)
Copied to clipboard
| Challenge: | We use tangrams as stimuli in cognitive science to study abstract visual reasoning . pre-trained weights demonstrate limited abstract reasoning, we observe . |
| Approach: | They propose a resource for studying abstract visual reasoning in humans and machines . they use tangram puzzles as stimuli to create an annotated dataset with >1k distinct stimuli . |
| Outcome: | The proposed resource is visually and linguistically richer than previous resources . pre-trained weights demonstrate limited abstract reasoning, the authors note . |
Simulating Bandit Learning from User Feedback for Extractive Question Answering (2022.acl-long)
Copied to clipboard
| Challenge: | Explicit feedback from users can be used to continually improve system performance. |
| Approach: | They study the potential of learning from user feedback for extractive question answering by simulating feedback using supervised data. |
| Outcome: | The proposed model improves on a few examples and can be deployed in new domains without any data annotation effort. |
Success and Cost Elicit Convention Formation for Efficient Communication (2026.acl-long)
Copied to clipboard
| Challenge: | ad hoc conventions allow people to coordinate on short, less costly utterances that are understood using shared conversational context. |
| Approach: | They propose a method to train large multimodal models to form conventions . they use simulated reference games to produce training data . |
| Outcome: | The proposed method reduces message length by up to 41% while increasing success by 15% over the course of the interaction. |
Continually Improving Extractive QA via Human Feedback (2023.emnlp-main)
Copied to clipboard
| Challenge: | a study of extractive question answering systems using human feedback shows promising potential for continual learning. |
| Approach: | They study extractive question answering system by using user feedback to improve it . they design and deploy an iterative approach where users ask questions and provide feedback . |
| Outcome: | The proposed model improves over time across different data regimes and domains . human user feedback is more affordable and abundant than annotations provided by trained experts . |
Interactive Classification by Asking Informative Questions (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for intent classification rely on a single user input and do not interact with the user to reduce ambiguity and improve the final prediction. |
| Approach: | They propose a limited form of interaction to natural language intent classification . they add binary or multi-choice questions to the system to ask missing information . |
| Outcome: | The proposed method can be bootstrapped without interaction data and is scalable to two domains. |
Evaluating Models’ Local Decision Boundaries via Contrast Sets (2020.findings-emnlp)
Copied to clipboard
Matt Gardner, Yoav Artzi, Victoria Basmov, Jonathan Berant, Ben Bogin, Sihao Chen, Pradeep Dasigi, Dheeru Dua, Yanai Elazar, Ananth Gottumukkala, Nitish Gupta, Hannaneh Hajishirzi, Gabriel Ilharco, Daniel Khashabi, Kevin Lin, Jiangming Liu, Nelson F. Liu, Phoebe Mulcaire, Qiang Ning, Sameer Singh, Noah A. Smith, Sanjay Subramanian, Reut Tsarfaty, Eric Wallace, Ally Zhang, Ben Zhou
| Challenge: | Standard test sets for supervised learning evaluate in-distribution generalization but are misleading when a dataset has systematic gaps. |
| Approach: | They propose a more rigorous annotation paradigm for NLP that helps to close systematic gaps in the test data. |
| Outcome: | The proposed model performs significantly lower on contrast sets than on the original test sets—up to 25% in some cases. |
Mapping Instructions to Actions in 3D Environments with Visual Goal Prediction (D18-1)
Copied to clipboard
| Challenge: | Existing models that map from inputs to actions are inefficient and require hand-crafted meaning representations. |
| Approach: | They propose to decompose instruction execution to goal prediction and action generation . they introduce two benchmarks for instruction following: LANI and CHAI . |
| Outcome: | The proposed model decomposes instruction execution to goal prediction and action generation. |
Analysis of Language Change in Collaborative Instruction Following (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work has found that language complexity is reduced along multiple dimensions as conventions are formed. |
| Approach: | They analyze language change over time in a collaborative task where utility-maximizing participants form conventions and increase their expertise. |
| Outcome: | The study shows that instructors increase language complexity along dimensions to collaborate with skill followers. |
CB2: Collaborative Natural Language Interaction Research Platform (2023.acl-demo)
Copied to clipboard
| Challenge: | CB2 is a platform to study collaborative grounded natural language interactions in task-oriented scenarios . it includes a 3D game environment, backend server, tools and processes to enable scalable studies. |
| Approach: | They propose a multi-agent platform to study collaborative natural language interactions in a task-oriented scenario. |
| Outcome: | The proposed model is a demonstration of a collaborative natural language agent . it is based on the CEREALBAR scenario, but is scalable to emphasize accessibility . |
Learning to Map Context-Dependent Sentences to Executable Formal Queries (N18-1)
Copied to clipboard
| Challenge: | Existing models that map utterances to executable queries are context-dependent and can incorporate interaction history. |
| Approach: | They propose a context-dependent model that maps utterances to executable queries . their approach combines implicit and explicit modeling of references between utterations . |
| Outcome: | The proposed model can map utterances to executable queries based on interaction history . key to mapping utterrances to queries is resolving references . |
A Corpus for Reasoning about Natural Language Grounded in Photographs (P19-1)
Copied to clipboard
| Challenge: | a dataset for visual reasoning with natural language and images is available. |
| Approach: | They propose a dataset for joint reasoning about natural language and images . they crowdsource 107,292 examples of English sentences paired with web photographs . |
| Outcome: | The proposed dataset combines 107,292 examples of English sentences with web photographs . Qualitative analysis shows the data requires compositional joint reasoning . |
When in Doubt: Improving Classification Performance with Alternating Normalization (2021.findings-emnlp)
Copied to clipboard
| Challenge: | a classifier that uses a nonparametric post-processing step for classification suffers when given examples that are close to its decision boundary. |
| Approach: | They propose a nonparametric post-processing step that re-adjusts predicted class probability distributions using high-confidence validation examples. |
| Outcome: | The proposed method improves classifier accuracy on difficult examples. |
What is Learned in Visually Grounded Neural Syntax Acquisition (2020.acl-main)
Copied to clipboard
| Challenge: | Visual features are promising for learning bootstrap textual models, but blackbox learning models make it difficult to isolate the specific contribution of visual components. |
| Approach: | They propose to use alignments between phrases and images as a learning signal for syntax acquisition. |
| Outcome: | The proposed model performs better than the previous model, but it is significantly less expressive. |
Retrospective Learning from Interactions (2025.acl-long)
Copied to clipboard
| Challenge: | Multi-turn interactions between large language models and users naturally include implicit feedback signals. |
| Approach: | They propose a method to learn from feedback signals in past interactions without annotations . they use a multimodal LLM to solve a reasoning task with a combinatorial solution space . |
| Outcome: | The proposed method improves task completion rate from 31% to 82% without annotations. |