Papers by Alane Suhr
Executing Instructions in Situated Collaborative Interactions (D19-1)
Copied to clipboard
Alane Suhr, Claudia Yan, Jack Schluger, Stanley Yu, Hadi Khader, Marwa Mouallem, Iris Zhang, Yoav Artzi
| Challenge: | a collaborative game with natural language instruction allows users to adapt to the system abilities by changing their language or deciding to accomplish tasks themselves. |
| Approach: | They propose a collaborative game where a user instructs a system to complete tasks, but acts alongside it. |
| Outcome: | The proposed game allows users to adapt to the system abilities by changing their language or deciding to accomplish tasks themselves. |
Continual Learning for Grounded Instruction Generation by Observing Human Following Behavior (2021.tacl-1)
Copied to clipboard
| Challenge: | Despite its potential and prevalence, this signal is understudied for learning to generate natural language. |
| Approach: | They propose to use this signal to improve the system's ability to generate instructions via contextual bandit learning. |
| Outcome: | The proposed system improves its ability to generate natural language through interaction with users, and the results are shown. |
Exploring Unexplored Generalization Challenges for Cross-Database Semantic Parsing (2020.acl-main)
Copied to clipboard
| Challenge: | Existing evaluation datasets such as Spider are used to support cross-database semantic parsing . XSP systems that map natural language utterances to SQL queries are evaluated on databases unseen during training. |
| Approach: | They propose a setup that uses eight well-studied datasets to evaluate cross-database semantic parsing systems. |
| Outcome: | The proposed system performs well on spider, but struggles to generalize to the repurposed set. |
Crowdsourcing Beyond Annotation: Case Studies in Benchmark Data Collection (2021.emnlp-tutorials)
Copied to clipboard
| Challenge: | Developing a theory of crowdsourcing for practical language problems remains an open challenge . |
| Approach: | This tutorial exposes NLP researchers to data collection crowdsourcing methods and principles through case studies. |
| Outcome: | This tutorial exposes NLP researchers to various data collection crowdsourcing methods and practices through case studies. |
Linear Script Representations in Speech Foundation Models Enable Zero-Shot Transliteration (2026.findings-acl)
Copied to clipboard
Ryan Soh-Eun Shim, Kwanghee Choi, Kalvin Chang, Ming-Hao Hsu, Florian Eichin, Zhizheng Wu, Alane Suhr, Michael A. Hedderich, David Harwath, David R. Mortensen, Barbara Plank
| Challenge: | We show that script information is linearly encoded in the activation space of multilingual speech models . modifying activations at inference time induces script change even in unconventional pairings . |
| Approach: | They propose to add script vectors to activations at test time to induce script change . they also show that script information is linearly encoded in the activation space of multilingual speech models . |
| Outcome: | The proposed approach can induce script change even in unconventional language-script pairings. |
Situated Mapping of Sequential Instructions to Actions with Single-step Reward Observation (P18-1)
Copied to clipboard
| Challenge: | Existing approaches to map context-dependent sequential instructions to actions are based on discourse and state dependencies . we evaluate on SCONE domains and show absolute accuracy improvements of 9.8%-25.3% . |
| Approach: | They propose a model that considers previous utterances and the state of the world to map sequential instructions to actions. |
| Outcome: | The proposed model improves on the SCONE domains and on the target domains. |
Abstract Visual Reasoning with Tangram Shapes (2022.emnlp-main)
Copied to clipboard
| Challenge: | We use tangrams as stimuli in cognitive science to study abstract visual reasoning . pre-trained weights demonstrate limited abstract reasoning, we observe . |
| Approach: | They propose a resource for studying abstract visual reasoning in humans and machines . they use tangram puzzles as stimuli to create an annotated dataset with >1k distinct stimuli . |
| Outcome: | The proposed resource is visually and linguistically richer than previous resources . pre-trained weights demonstrate limited abstract reasoning, the authors note . |
Grounding Language in Multi-Perspective Referential Communication (2024.emnlp-main)
Copied to clipboard
| Challenge: | Using a dataset of 2,970 human-written referring expressions, we find that the performance of automated models in both reference generation and comprehension lags behind that of pairs of human agents. |
| Approach: | They propose a task and dataset for referring expression generation and comprehension in multi-agent embodied environments where two agents must take into account one another's visual perspective to produce and understand references to objects in a scene. |
| Outcome: | The proposed model outperforms the strongest proprietary model and improves communicative success from 58.9 to 69.3% when trained with a listener. |
Using Language Models to Disambiguate Lexical Choices in Translation (2024.emnlp-main)
Copied to clipboard
| Challenge: | In translation, a concept represented by a single word can have multiple variations in a target language. |
| Approach: | They evaluate language models that can be used to generate English rules for lexical selection . they find weaker models with high-quality lexicals improve accuracy . |
| Outcome: | The proposed model outperforms existing models on the lexical selection task in English and with native speakers. |
Minding Language Models’ (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Tracker (2023.acl-long)
Copied to clipboard
| Challenge: | Empirical results show plug-and-play approach to reason about belief states of multiple characters in reading comprehension tasks is more precise and interpretable than previous approaches. |
| Approach: | They propose a plug-and-play approach to reason about the belief states of multiple characters in reading comprehension tasks via explicit symbolic representation. |
| Outcome: | The proposed algorithm improves theory of mind of off-the-shelf neural language models without supervision. |
UNcommonsense Reasoning: Abductive Reasoning about Uncommon Situations (2024.naacl-long)
Copied to clipboard
Wenting Zhao, Justin Chiu, Jena Hwang, Faeze Brahman, Jack Hessel, Sanjiban Choudhury, Yejin Choi, Xiang Li, Alane Suhr
| Challenge: | Existing work evaluating commonsense reasoning focuses on making inferences about common, everyday situations. |
| Approach: | They propose to use an English language corpus to investigate commonsense reasoning . they characterize performance differences between human explainers and best-performing large language models . |
| Outcome: | The proposed method reduces the loss rate of human-written explanations on commonsense reasoning compared with the vanilla supervised fine-tuning approach . |
Analysis of Language Change in Collaborative Instruction Following (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work has found that language complexity is reduced along multiple dimensions as conventions are formed. |
| Approach: | They analyze language change over time in a collaborative task where utility-maximizing participants form conventions and increase their expertise. |
| Outcome: | The study shows that instructors increase language complexity along dimensions to collaborate with skill followers. |
Learning to Map Context-Dependent Sentences to Executable Formal Queries (N18-1)
Copied to clipboard
| Challenge: | Existing models that map utterances to executable queries are context-dependent and can incorporate interaction history. |
| Approach: | They propose a context-dependent model that maps utterances to executable queries . their approach combines implicit and explicit modeling of references between utterations . |
| Outcome: | The proposed model can map utterances to executable queries based on interaction history . key to mapping utterrances to queries is resolving references . |
A Corpus for Reasoning about Natural Language Grounded in Photographs (P19-1)
Copied to clipboard
| Challenge: | a dataset for visual reasoning with natural language and images is available. |
| Approach: | They propose a dataset for joint reasoning about natural language and images . they crowdsource 107,292 examples of English sentences paired with web photographs . |
| Outcome: | The proposed dataset combines 107,292 examples of English sentences with web photographs . Qualitative analysis shows the data requires compositional joint reasoning . |
We’re Afraid Language Models Aren’t Modeling Ambiguity (2023.emnlp-main)
Copied to clipboard
Alisa Liu, Zhaofeng Wu, Julian Michael, Alane Suhr, Peter West, Alexander Koller, Swabha Swayamdipta, Noah Smith, Yejin Choi
| Challenge: | Ambiguity is an intrinsic feature of natural language, allowing us to anticipate misunderstandings and revise our interpretations as listeners. |
| Approach: | They use AmbiEnt to capture ambiguity in a sentence and analyze it to evaluate pretrained LMs. |
| Outcome: | The proposed model can flag political claims in the wild that are misleading due to ambiguity. |
Neural Semantic Parsing (P18-5)
Copied to clipboard
| Challenge: | Semantic parsing is the study of translating natural language utterances into machine-executable programs. |
| Approach: | They will describe the various approaches researchers have taken to translate natural language into a formal language . they will also discuss why much recent work has chosen to use standard programming languages instead of more linguistically-motivated representations. |
| Outcome: | This paper will describe the various approaches researchers have taken to translate natural language into a formal language. |