Papers by Elias Stengel-Eskin
Did You Mean...? Confidence-based Trade-offs in Semantic Parsing (2023.emnlp-main)
Copied to clipboard
| Challenge: | a calibrated model can help balance common trade-offs in task-oriented parsing. |
| Approach: | They propose a model which rephrases low-confidence inputs to improve usability and safety. |
| Outcome: | The proposed system reduces the number of incorrect low-confidence programs executed, but at a cost to usability. |
Teaching Models to Balance Resisting and Accepting Persuasion (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are susceptible to persuasion, which can pose risks when faced with an adversarial interlocutor. |
| Approach: | They propose a method to balance positive and negative persuasion by using recursive dialogue trees to train models to accept persulasion. |
| Outcome: | The proposed model-based training improves resistance to misinformation and resilience to being challenged while also resulting in the best overall performance on multi-agent debates across two domains. |
DART: Leveraging Multi-Agent Disagreement for Tool Recruitment in Multimodal Reasoning (2026.eacl-long)
Copied to clipboard
Nithin Sivakumaran, Justin Chen, David Wan, Yue Zhang, Jaehong Yoon, Elias Stengel-Eskin, Mohit Bansal
| Challenge: | a key strength of human intelligence is the ability to debate and discuss reasoning with others. |
| Approach: | They propose a multi-agent framework that uses disagreements between visual agents to identify useful visual tools that can resolve inter-agency disagreement. |
| Outcome: | The proposed framework beats the strongest baseline on A-OKVQA and MMMU, respectively. |
Multi-Attribute Steering of Language Models via Targeted Intervention (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches for steering large language models fail to scale to multi-attribute settings with conflicts, such as enhancing helpfulness while also reducing toxicity. |
| Approach: | They propose a steering framework for selective token-level intervention across multiple attributes that enforcing sparsity and orthogonality among vectors for different attributes. |
| Outcome: | The proposed framework outperforms existing ITI and parameter-efficient fine-tuning approaches across question answering tasks and generative tasks. |
AdaCAD: Adaptively Decoding to Balance Conflicts between Contextual and Parametric Knowledge (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing contrastive methods that ignore the context of a large language model (LLM) fail to handle instances that vary in their amount of conflict, with static methods over-adjusting when conflict is absent. |
| Approach: | They propose a fine-grained, instance-level approach called AdaCAD which dynamically adjusts the degree of conflict based on the degree. |
| Outcome: | The proposed approach outperforms baselines and improves factuality of summaries by 6.19. |
Joint Universal Syntactic and Semantic Parsing (2021.tacl-1)
Copied to clipboard
| Challenge: | Several attempts have been made to jointly parse syntax and semantics, but this trade-off is not well understood. |
| Approach: | They propose multiple model architectures that exploit the rich syntactic and semantic annotations contained in the Universal Decompositional Semantics dataset to obtain state-of-the-art results. |
| Outcome: | The proposed model outperforms existing models in 8 languages and their results are consistent across languages. |
PRInTS: Reward Modeling for Long-Horizon Information Seeking (2026.acl-long)
Copied to clipboard
| Challenge: | Existing PRMs cannot capture richer dimensions of information-seeking steps, such as tool interactions and reasoning over tool outputs. |
| Approach: | They propose a generative PRM trained with dual capabilities that compresses the growing context while preserving essential information for step evaluation. |
| Outcome: | PRInTS improves on FRAMES, GAIA, and WebWalkerQA models while preserving essential information for step evaluation. |
A Discriminative Neural Model for Cross-Lingual Word Alignment (D19-1)
Copied to clipboard
| Challenge: | a novel word alignment model for machine translation has been developed for a number of languages . explicit word-to-word alignments have largely been lost in neural MT systems . |
| Approach: | They propose a discriminative word alignment model which integrates into a Transformer-based machine translation model. |
| Outcome: | The proposed model performs better on Chinese and Arabic alignments than standard models. |
Routing with Generated Data: Annotation-Free LLM Skill Estimation and Expert Selection (2026.acl-long)
Copied to clipboard
Tianyi Niu, Justin Chen, Genta Indra Winata, Shi-Xiong Zhang, Supriyo Chakraborty, Sambit Sahu, Yue Zhang, Elias Stengel-Eskin, Mohit Bansal
| Challenge: | Existing approaches typically assume access to ground-truth labeled data . Existing methods require a classifier to select models given an input . |
| Approach: | They propose a routing setting where routers are trained exclusively on generated queries and answers from LLMs. |
| Outcome: | The proposed router outperforms the best query-answer router by 4.6% absolute accuracy when trained on weak generator data. |
Soft Self-Consistency Improves Language Models Agents (2024.acl-short)
Copied to clipboard
| Challenge: | Current “sample and select” methods rely on majority voting to score answers . however, when tasks have many distinct and valid answers, selection by voting requires a large number of samples. |
| Approach: | They introduce a method that replaces SC's discontinuous scoring with a continuous score computed from model likelihoods to increase selection even when actions are sparsely distributed. |
| Outcome: | The proposed method improves performance and efficiency on long-horizon interactive tasks by replacing SC’s discontinuous scoring with a continuous score computed from model likelihoods. |
Visual Commonsense in Pretrained Unimodal and Multimodal Models (2022.naacl-main)
Copied to clipboard
| Challenge: | Fig. 1 shows how text-only and image-only models can capture commonsense visual attributes, but reporting bias affects their performance. |
| Approach: | They use a Visual Commonsense Tests dataset to validate their findings . they find multimodal models better reconstruct attribute distributions, but are still subject to reporting bias . |
| Outcome: | The proposed model improves on the unimodal and multimodal models, but is still subject to reporting bias. |
Calibrated Interpretation: Confidence Estimation in Semantic Parsing (2023.tacl-1)
Copied to clipboard
| Challenge: | Sequence generation models are increasingly being used to translate natural language into programs . calibration of such models is a key component of safety, says aaron sagar . |
| Approach: | They investigate whether calibration of popular generation models varies across models and datasets . they find that calibration varies among models and data sets, and that it is important to include it in evaluations if it is included . |
| Outcome: | The calibration of popular generation models varies across models and datasets . the authors find that the accuracy of models is dependent on confidence . |
LAQuer: Localized Attribution Queries in Content-grounded Generation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing attribution methods associate entire sentences with source documents, which can be overwhelming for users seeking to fact-check specific claims. |
| Approach: | They propose a task that localizes selected spans of generated output to their corresponding source spans, allowing fine-grained and user-directed attribution. |
| Outcome: | The proposed task localizes selected spans of generated output to their corresponding source spans, allowing fine-grained and user-directed attribution. |
MAMM-Refine: A Recipe for Improving Faithfulness in Generation with Multi-Agent Collaboration (2025.naacl-long)
Copied to clipboard
| Challenge: | Multi-agent collaboration among models has shown promise in reasoning tasks but is underexplored in long-form generation tasks like summarization and question-answering. |
| Approach: | They propose a multi-agent multi-model reasoning recipe to improve faithfulness through refinement. |
| Outcome: | The proposed method improves faithfulness and error detection on three summarization datasets and on long-form question-answering tasks. |
RotBench: Evaluating Multi-modal Large Language Models on Identifying Image Rotation (2026.eacl-long)
Copied to clipboard
| Challenge: | Multimodal Large Language Models (MLLMs) can identify the orientation of input images rotated 0°, 90°, 180°, and 270°. |
| Approach: | They propose a manually-filtered benchmark to evaluate MLLMs' ability to accurately identify rotation in input images. |
| Outcome: | The proposed model improves on the 'rotational cues' of 360° and 180° images, but not 90° and 270° rotations. |
Language Models Identify Ambiguities and Exploit Loopholes (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing models that exploit loopholes identify and reason about ambiguity and conflicting goals, presenting a potential safety risk. |
| Approach: | They propose to study the responses of large language models to loopholes by examining ambiguity and pragmatics in LLMs. |
| Outcome: | The proposed models can identify ambiguities and exploit loopholes to satisfy their given goals as opposed to the goals of the user. |
The Curious Case of Control (2022.emnlp-main)
Copied to clipboard
| Challenge: | Normally-developing children struggle with subject control clauses long after they have acquired the components to understand them. |
| Approach: | They examine whether heuristics based on semantic roles are consistent with children's English . they find that models are more sensitive to agent-patient information . |
| Outcome: | The results show that models fail on subject control but fail on object control . the authors show that raising salience of agent and patient relations results in significant changes . |
When More Data Hurts: A Troubling Quirk in Developing Broad-Coverage Natural Language Understanding Systems (2022.emnlp-main)
Copied to clipboard
Elias Stengel-Eskin, Emmanouil Antonios Platanios, Adam Pauls, Sam Thomson, Hao Fang, Benjamin Van Durme, Jason Eisner, Yu Su
| Challenge: | In natural language understanding systems, users’ evolving needs necessitate the addition of new features over time, indexed by new symbols added to the meaning representation space. |
| Approach: | They propose to use a small set of new symbols to build broad-coverage NLU systems. |
| Outcome: | The proposed model is based on two prototypical NLU tasks: intent recognition and semantic parsing. |
GrAInS: Gradient-based Attribution for Inference-Time Steering of LLMs and VLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning large language models often ignore token-level causal influence and underutilize model logits. |
| Approach: | They propose a novel approach that uses a gradient-based approach to identify influential tokens and construct directional steering vectors based on their contribution to preferred over dispreferred outputs. |
| Outcome: | The proposed approach outperforms fine-tuning and prior steering methods on both LLM and VLM tasks without degrading fluency or general capabilities. |
MAgICoRe: Multi-Agent, Iterative, Coarse-to-Fine Refinement for Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Excessive refinement can cause over-correction and reduce performance, authors say . they say MAgICoRe is a framework for multi-agent iteration for coarse-to-fine refinement . |
| Approach: | They propose a framework for multi-agent iteration for coarse-to-fine refinement that reduces excessive refinement by categorizing problems as easy or hard. |
| Outcome: | The proposed framework beats Self-Consistency by 3.4%, Best-of-k by 3.2%, and Self-Refine by 4.0% on Llama-3-8B and GPT- 3.5. |
Iterative Paraphrastic Augmentation with Discriminative Span Alignment (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing datasets can be expanded or created using a small, manually produced seed corpus. |
| Approach: | They propose a paraphrastic augmentation strategy based on sentence-level lexically constrained paraphrases and discriminative span alignment. |
| Outcome: | The proposed approach allows for the large-scale expansion of existing datasets or the rapid creation of new datasets using a small, manually produced seed corpus. |
Universal Decompositional Semantic Parsing (2020.acl-main)
Copied to clipboard
| Challenge: | Decompositional Semantics is a formalism that encodes semantic information in a feature-based scheme using continuous scales rather than categorical labels. |
| Approach: | They propose a transductive model for parsing into Universal Decompositional Semantics representations and a pipeline model for annotating the graph with decompositionally semantic attribute scores. |
| Outcome: | The proposed model performs well while performing attribute prediction. |
Instruction Tuning with and without Context: Behavioral Shifts and Downstream Impact (2026.eacl-long)
Copied to clipboard
Hyunji Lee, Seunghyun Yoon, Yunjae Won, Hanseok Oh, Geewook Kim, Trung Bui, Franck Dernoncourt, Elias Stengel-Eskin, Mohit Bansal, Minjoon Seo
| Challenge: | Prior work on instruction tuning datasets combined these data types without examining their distinct effects. |
| Approach: | They investigate how training LLMs with or without context affects model behavior and performance . they find that using context-augmented data as the backbone for vision-language models reduces hallucination . |
| Outcome: | The proposed training with context-augmented data reduces hallucination and improves grounding in the visual domain. |
Why Did the Chicken Cross the Road? Rephrasing and Analyzing Ambiguous Questions in VQA (2023.acl-long)
Copied to clipboard
| Challenge: | Visual question answering models seek to answer questions about images . ambiguity can exist at all levels of linguistic analysis, but disagreements can be difficult to detect and resolve . |
| Approach: | They develop a question-generation model which integrates group information without supervision and uses a dataset of ambiguous examples to annotate answers. |
| Outcome: | The proposed model can integrate answer group information without supervision and is able to fill knowledge gaps and convey requests. |
The Universal Decompositional Semantics Dataset and Decomp Toolkit (2020.lrec-1)
Copied to clipboard
Aaron Steven White, Elias Stengel-Eskin, Siddharth Vashishtha, Venkata Subrahmanyan Govindarajan, Dee Ann Reisinger, Tim Vieira, Keisuke Sakaguchi, Sheng Zhang, Francis Ferraro, Rachel Rudinger, Kyle Rawlins, Benjamin Van Durme
| Challenge: | Decompositional semantics is a method of crowd-sourcing semantic annotations while retaining high interannotator agreement. |
| Approach: | They present the Universal Decompositional Semantics dataset (v1.0) they propose a decomposition-aligned approach to semantic annotation that uses simple questions to answer . |
| Outcome: | The dataset is bundled with the Decomp toolkit (v0.1) both datasets are publicly available at http://decomp.io. |