Papers by Jason Eisner

Unsupervised Disambiguation of Syncretism in Inflected Lexicons (N18-2)

Challenge:	Existing approaches to task-oriented dialogue represent dialogue state as a dataflow graph . microsoft's SMCalFlow dataset features complex dialogues about events, weather, places, and people .
Approach:	They propose a dataflow graph-based dialogue agent that maps each user utterance to a program that extends this graph.
Outcome:	The proposed framework improves representability and predictability in natural dialogues . it uses dataflow graphs and metacomputation to map user intents to a program .

Copied to clipboard

Ryan Cotterell, Christo Kirov, Sabrina J. Mielke, Jason Eisner

Language-to-Code Translation with a Single Labeled Example (2024.emnlp-main)

Challenge:	Lexical ambiguity makes it difficult to compute useful statistics of a corpus.
Approach:	They propose a neural network-based model that fits a prior distribution over feature bundles to a list of unigram type counts and partitions each count among different analyses of that unigrammer.
Outcome:	The proposed model is based on a list of unigram type counts and partitions each count among different analyses of that unigrammer.

Copied to clipboard

Kaj Bostrom, Harsh Jhamtani, Hao Fang, Sam Thomson, Richard Shin, Patrick Xia, Benjamin Van Durme, Jason Eisner, Jacob Andreas

Text Generation

Challenge:	In-Context Inverse Programming (ICIP) bootstraps a language-to-code system using mostly unlabeled programs written using a potentially unfamiliar library or API.
Approach:	They propose a method for bootstrapping a language-to-code system using mostly unlabeled programs written using a potentially unfamiliar library or API.
Outcome:	The proposed method outperforms baselines across nine domains and 100 examples in a “nearly unsupervised” setting.

Searching for More Efficient Dynamic Programs (2021.findings-emnlp)

Copied to clipboard

Tim Vieira, Ryan Cotterell, Jason Eisner

When is a Language Process a Language Model? (2024.findings-acl)

Challenge:	a combinatorial model of human language often involves dynamic programming.
Approach:	They propose to search for a sequence of semantics-preserving transformations to improve the initial program's running time.
Outcome:	The proposed algorithm can find speed-ups in the initial program, and it can be used to improve it.

Copied to clipboard

Li Du, Holden Lee, Jason Eisner, Ryan Cotterell

The Whole Truth and Nothing But the Truth: Faithful and Controllable Dialogue Response Generation with Dataflow Transduction and Constrained Decoding (2023.findings-acl)

Challenge:	In some pathological situations, such a stochastic process may "leak" probability mass onto the set of infinite strings.
Approach:	They propose to view a language model as a discrete stochastic process X t : t = = t + .
Outcome:	The proposed conditions of tightness are generalized to language models and the literature.

Copied to clipboard

Hao Fang, Anusha Balakrishnan, Harsh Jhamtani, John Bufe, Jean Crawford, Jayant Krishnamurthy, Adam Pauls, Jason Eisner, Jacob Andreas, Dan Klein

Dialogue Systems & Conversational Agents

Challenge:	In a task-oriented dialogue system, response generation is a conditional language model, but effective dialogue agents must balance fluent generation with stricter constraints.
Approach:	They propose a rule-based content selection model that transduces a dialogue agent’s actions and their results into context-free grammars representing the space of contextually acceptable responses.
Outcome:	The proposed architecture outperforms both rule-based and learned approaches in human evaluations of fluency, relevance, and truthfulness.

LLMs Know More About Numbers than They Can Say (2026.eacl-short)

Copied to clipboard

Fengting Yuchi, Li Du, Jason Eisner

Numerical Reasoning

Challenge:	Large language models (LLMs) are increasingly used in mathematical, scientific, financial and engineering domains.
Approach:	They probe the hidden states of several smaller open-source LLMs to find out how big they are .
Outcome:	The proposed model improves verbalized accuracy by 3.22% over base models.

What Kind of Language Is Hard to Language-Model? (P19-1)

Copied to clipboard

Sabrina J. Mielke, Ryan Cotterell, Kyle Gorman, Brian Roark, Jason Eisner

Algorithms for Acyclic Weighted Finite-State Automata with Failure Arcs (2022.emnlp-main)

Challenge:	a recent study suggests that language models perform poorly across languages.
Approach:	They propose a model that fits a paired-sample multiplicative mixed-effects model to obtain language difficulty coefficients from at least-pairwise parallel corpora.
Outcome:	The proposed model is able to handle missing data and is aware of inter-sentence variation.

Copied to clipboard

Anej Svete, Benjamin Dayan, Ryan Cotterell, Tim Vieira, Jason Eisner

Challenge:	Weighted finite-state automata (WSFAs) are common formalisms in NLP.
Approach:	They propose an algorithm for semiring-weighted WFSAs with av-erage out symbol fractions .
Outcome:	The proposed algorithms are faster than the standard methods for weighted finite-state automata.

Online Semantic Parsing for Latency Reduction in Task-Oriented Dialogue (2022.acl-long)

Copied to clipboard

Jiawei Zhou, Jason Eisner, Michael Newman, Emmanouil Antonios Platanios, Sam Thomson

Synthetic Data Made to Order: The Case of Parsing (D18-1)

Challenge:	Standard conversational semantic parsing maps a user's intent into an executable program, but execution is slow when expensive function calls are included.
Approach:	They propose a task of online semantic parsing to predict and execute function calls while the user is still speaking.
Outcome:	The proposed approach reduces latency with good parsing quality and execution cost.

Copied to clipboard

Dingquan Wang, Jason Eisner

Limitations of Autoregressive Models and Their Alternatives (2021.naacl-main)

Challenge:	supervised dependency parsing is a core task in natural language processing, but unsupervised parsers can hardly produce useful parses.
Approach:	They propose to permute the constituents of an existing dependency treebank so that its surface part-of-speech statistics approximately match those of the target language.
Outcome:	The proposed method improves the parsing accuracy of a target language . the proposed method is based on a distribution of gold POS bigrams .

Copied to clipboard

Chu-Cheng Lin, Aaron Jaech, Xin Li, Matthew R. Gormley, Jason Eisner

Interpreting User Requests in the Context of Natural Language Standing Instructions (2024.findings-naacl)

Challenge:	Standard autoregressive language models only perform polynomial-time computation to compute probability of next symbol.
Approach:	authors propose alternative to standard autoregressive language models that use polynomial-time computation to compute probability of next symbol.
Outcome:	a large model size can grow superpolynomially in length, allowing it to store precomputed results and verify solutions.

Copied to clipboard

Nikita Moghe, Patrick Xia, Jacob Andreas, Jason Eisner, Benjamin Van Durme, Harsh Jhamtani

Accelerating Language Model Workflows with Prompt Choreography (2026.tacl-1)

Challenge:	Existing approaches to LLM-based dialogue modeling provide additional context for users to make requests.
Approach:	They propose an approach to LLM-based dialogue modeling where persistent user constraints and preferences are provided as additional context for such interfaces.
Outcome:	The proposed model achieves a maximum of 46% exact match on the prediction of 2.4K English dialogues with a language-to-program dataset.

Copied to clipboard

TJ Bai, Jason Eisner

A Deep Generative Model of Vowel Formant Typology (N18-1)

Challenge:	Large language models are increasingly deployed in multi-agent workflows that require multiple agents to encode the same prompt from scratch.
Approach:	They propose a framework that maintains a dynamic, global KV cache that allows agents to attend to arbitrary, reordered subsets of previously encoded messages.
Outcome:	The proposed framework significantly reduces per-message latency (2.0–6.2 faster time-to-first-token) and achieves substantial speedups (>2.2) in some workflows dominated by redundant computation.

Copied to clipboard

Ryan Cotterell, Jason Eisner

Contrastive Decoding: Open-ended Text Generation as Optimization (2023.acl-long)

Challenge:	a recent study has investigated the nature of vowel inventories, i.e., which vowels a language contains . a probabilistic approach does not rule out linguistic systems completely, but it can position phenomena on a scale from very common to very improbable.
Approach:	They propose a generative probability model of vowel inventory typology based on acoustic information rather than discrete symbols from the international phonetic alphabet.
Outcome:	The proposed model uses acoustic information rather than discrete symbols from the phonetic alphabet.

Copied to clipboard

Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori Hashimoto, Luke Zettlemoyer, Mike Lewis

Do Androids Know They’re Only Dreaming of Electric Sheep? (2024.findings-acl)

Challenge:	Using a language model, maximum probability is a poor decoding objective because it produces short and repetitive text.
Approach:	They propose a reliable decoding approach that optimizes a contrastive objective subject to a plausibility constraint.
Outcome:	The proposed approach outperforms four strong decoding algorithms in automatic and human evaluations across wikipedia, news and story domains.

Copied to clipboard

Sky CH-Wang, Benjamin Van Durme, Jason Eisner, Chris Kedzie

Spelling-Aware Construction of Macaronic Texts for Teaching Foreign-Language Vocabulary (D19-1)

Challenge:	Detecting hallucinations in grounded generation tasks is commonly framed as a textual entailment problem.
Approach:	They develop probes that are narrowly trained to predict hallucination in a transformer language model.
Outcome:	The probes can detect hallucinations at many transformer layers outperforming baselines and human annotators on two out of three generation tasks.

Copied to clipboard

Adithya Renduchintala, Philipp Koehn, Jason Eisner

On the Intersection of Context-Free and Regular Languages (2023.eacl-main)

Challenge:	a machine foreign-language teacher replaces word tokens with glosses in a foreign language to ease the human reader into understanding the L2 vocabulary.
Approach:	They propose a machine foreign-language teacher that modifies text by replacing word tokens with glosses in a foreign language to ease the human reader into understanding the L2 .
Outcome:	The proposed model can learn representations for novel words and is a proxy for word guessing and learning ability of real human students.

Copied to clipboard

Clemente Pasti, Andreas Opedal, Tiago Pimentel, Tim Vieira, Jason Eisner, Ryan Cotterell

Linguistic Theories

Challenge:	We generalize the Bar-Hillel intersection construction so that the given WFSA may contain -arcs.
Approach:	They propose a construction that generalizes the Bar- Hillel in the case the desired automaton has -arcs and generalize the weighted extension so that the given WFSA may contain arcs.
Outcome:	The proposed construction can encode the structure of both the input automaton and grammar while retaining the asymptotic size of the original construction.

A Glitch in the Matrix? Locating and Detecting Language Model Grounding with Fakepedia (2024.acl-long)

Copied to clipboard

Giovanni Monea, Maxime Peyrard, Martin Josifoski, Vishrav Chaudhary, Jason Eisner, Emre Kiciman, Hamid Palangi, Barun Patra, Robert West

MICE for CATs: Model-Internal Confidence Estimation for Calibrating Agents with Tools (2025.naacl-long)

Challenge:	Large language models (LLMs) have an impressive ability to draw on novel information supplied in their context, yet the mechanisms underlying contextual grounding remain unknown.
Approach:	They propose a method to study grounding abilities using a counterfactual dataset constructed to clash with a model's parametric knowledge using Fakepedia.
Outcome:	The proposed method evaluates grounding abilities when the internal parametric knowledge clashes with the contextual information.

Copied to clipboard

Nishant Subramani, Jason Eisner, Justin Svegliato, Benjamin Van Durme, Yu Su, Sam Thomson

Challenge:	Existing models that measure confidence in tool-using agents are poorly calibrated.
Approach:	They propose a class of model-internal confidence estimators to assess confidence when calling tools.
Outcome:	The proposed class beats baselines on a simulated trial and error (STE) tool-calling dataset using Llama3 models.

Constrained Language Models Yield Few-Shot Semantic Parsers (2021.emnlp-main)

Copied to clipboard

Richard Shin, Christopher Lin, Sam Thomson, Charles Chen, Subhro Roy, Emmanouil Antonios Platanios, Adam Pauls, Dan Klein, Jason Eisner, Benjamin Van Durme

UniMorph 2.0: Universal Morphology (L18-1)

Challenge:	Large pretrained language models excel at generating natural language, but they are not efficient for task specific semantic parsing.
Approach:	They propose to use large pretrained language models as few-shot semantic parsers . they paraphrase inputs into a controlled sublanguage resembling English .
Outcome:	The proposed model can generate surprisingly accurate models on multiple tasks with minimal code and data.

Copied to clipboard

Christo Kirov, Ryan Cotterell, John Sylak-Glassman, Géraldine Walther, Ekaterina Vylomova, Patrick Xia, Manaal Faruqui, Sabrina J. Mielke, Arya McCarthy, Sandra Kübler, David Yarowsky, Jason Eisner, Mans Hulden

Specializing Word Embeddings (for Parsing) by Information Bottleneck (D19-1)

Challenge:	The Universal Morphology project is a collaborative effort to improve how NLP handles complex morphology across the world's languages.
Approach:	They propose to use a universal tagset to annotate morphological data using a schema that includes a lemma and a bundle of morphology features.
Outcome:	The project releases annotated morphological data using a universal tagset, the UniMorph schema.

Copied to clipboard

Xiang Lisa Li, Jason Eisner

Representation Learning

Challenge:	Pre-trained word embeddings contain rich syntactic and semantic information.
Approach:	They propose a variational information bottleneck method to nonlinearly compress word embeddings . they compress each word embedded to either a discrete tag or a continuous vector .
Outcome:	The proposed method can be extended to other syntactic or semantic tasks.

A Corpus for Large-Scale Phonetic Typology (2020.acl-main)

Copied to clipboard

Elizabeth Salesky, Eleanor Chodroff, Tiago Pimentel, Matthew Wiesner, Ryan Cotterell, Alan W Black, Jason Eisner

Are All Languages Equally Hard to Language-Model? (N18-2)

Challenge:	Existing multilingual speech corpora have limited data in many languages . existing corpus is limited to a small number of languages with available data .
Approach:	They propose a large-scale phonetic typology corpus with phoneme-level labels and phoneme alignments in 690 readings spanning 635 languages.
Outcome:	The proposed corpus covers 635 languages and includes acoustic-phonetic measures of vowels and sibilants.

Copied to clipboard

Ryan Cotterell, Sabrina J. Mielke, Jason Eisner, Brian Roark

Neural Particle Smoothing for Sampling from Conditional Sequence Models (N18-1)

Challenge:	a fair comparison of language models is tricky because of the size of the corpora and the variability of orthographic systems.
Approach:	They propose a framework for fair cross-linguistic comparison of language models . they show that in some languages, textual expression is harder to predict with n-gram models compared to LSTM models based on translated text .
Outcome:	The proposed framework is based on translated text and language models on 21 languages.

Copied to clipboard

Chu-Cheng Lin, Jason Eisner

Challenge:	In structured prediction problems, labeling an input string with a length-T sequence of tags becomes intractable.
Approach:	They propose a sequential Monte Carlo method for sampling annotations of an input string from a given probability model.
Outcome:	The proposed method improves the quality of the sample.

Learning to Retrieve Iteratively for In-Context Learning (2024.emnlp-main)

Copied to clipboard

Yunmo Chen, Tongfei Chen, Harsh Jhamtani, Patrick Xia, Richard Shin, Jason Eisner, Benjamin Van Durme

Neural Finite-State Transducers: Beyond Rational Relations (N19-1)

Challenge:	In-context learning is a powerful tool for learning large language models.
Approach:	They propose an iterative retrieval framework that empowers retrievers to make iterable decisions through policy optimization.
Outcome:	The proposed framework outperforms existing methods on semantic parsing datasets with 4M additional parameters for state encoding.

Copied to clipboard

Chu-Cheng Lin, Hao Zhu, Matthew R. Gormley, Jason Eisner

Challenge:	a finite state transducer defines joint and conditional probability distributions over strings . a weighted finite-state transducers can only model certain functions, known as the rational relations .
Approach:	They propose a family of string transduction models defining joint and conditional probability distributions over pairs of strings.
Outcome:	The proposed models are more powerful than previous finite-state models with neural features.

Learning How to Ask: Querying LMs with Mixtures of Soft Prompts (2021.naacl-main)

Copied to clipboard

Guanghui Qin, Jason Eisner

Toward Interactive Dictation (2023.acl-long)

Challenge:	Pretrained language models retain factual knowledge that can be extracted with a sentential prompt.
Approach:	They propose to learn prompts by gradient descent, either fine-tuning prompts or starting from random initialization.
Outcome:	The proposed approach outperforms existing methods on English LMs and tasks.

Copied to clipboard

Belinda Z. Li, Jason Eisner, Adam Pauls, Sam Thomson

Non-Programmers Can Label Programs Indirectly via Active Examples: A Case Study with Text-to-SQL (2023.emnlp-main)

Challenge:	Existing systems that allow both dictation and editing-by-voice restrict their command language to flat templates invoked by trigger words.
Approach:	They propose to allow users to interrupt dictation with spoken editing commands in open-ended natural language.
Outcome:	The proposed system can predict edited text with large pre-trained models and predict small programs.

Copied to clipboard

Ruiqi Zhong, Charlie Snell, Dan Klein, Jason Eisner

Programming Languages in NLP

Challenge:	Using APEL, non-programmers can annotate natural language utterances with complex programs that represent their meaning.
Approach:	They introduce a framework in which non-programmers select among candidate programs generated by a seed semantic parser.
Outcome:	The proposed framework achieves the same annotation accuracy as the original expert annotators (75%) and exposes many subtle errors in the original annotations.

When More Data Hurts: A Troubling Quirk in Developing Broad-Coverage Natural Language Understanding Systems (2022.emnlp-main)

Copied to clipboard

Elias Stengel-Eskin, Emmanouil Antonios Platanios, Adam Pauls, Sam Thomson, Hao Fang, Benjamin Van Durme, Jason Eisner, Yu Su

Challenge:	In natural language understanding systems, users’ evolving needs necessitate the addition of new features over time, indexed by new symbols added to the meaning representation space.
Approach:	They propose to use a small set of new symbols to build broad-coverage NLU systems.
Outcome:	The proposed model is based on two prototypical NLU tasks: intent recognition and semantic parsing.

LLMs in the Imaginarium: Tool Learning through Simulated Trial and Error (2024.acl-long)

Copied to clipboard

Boshi Wang, Hao Fang, Jason Eisner, Benjamin Van Durme, Yu Su

Time-and-Space-Efficient Weighted Deduction (2023.tacl-1)

Challenge:	Existing work on tool-augmented LLMs focuses on the broad coverage of tools and the flexibility of adding new tools.
Approach:	They propose a biologically inspired method for tool-augmented LLMs that orchestrates three key mechanisms for successful tool use behaviors in the biological system: trial and error, imagination, and memory.
Outcome:	The proposed method improves tool learning for LLMs under both in-context learning and fine-tuning settings, bringing a boost of 46.7% to Mistral-Instruct-7B and outperforms GPT-4.

Copied to clipboard

Jason Eisner

Reasoning

Challenge:	Unweighted deduction allows a generic forward-chaining execution strategy, but weighted deduction requires a constant factor more time and space.
Approach:	They propose a generic unweighted deduction strategy that uses a factor more time and space than unweighting deduction . they also propose an extension to cyclic deduction systems based on Tarjan .
Outcome:	The proposed method is based on the proposed method and is compared with cyclic deduction systems.

Contextualization of Morphological Inflection (N19-1)

Copied to clipboard

Ekaterina Vylomova, Ryan Cotterell, Trevor Cohn, Timothy Baldwin, Jason Eisner

Morphology

Challenge:	In this paper, we isolate the task of predicting a fully inflected sentence from its partially lemmatized version.
Approach:	They propose a task that requires morphological features to be inferred from sentential context . they propose morphology-based models that explicitly reconstruct morphologic features before predicting inflected forms .
Outcome:	The proposed model is able to predict inflected sentences without relying on morphological annotations.

Efficient Semiring-Weighted Earley Parsing (2023.acl-long)

Copied to clipboard

Andreas Opedal, Ran Zmigrod, Tim Vieira, Ryan Cotterell, Jason Eisner

Privacy-Preserving Domain Adaptation of Semantic Parsers (2023.acl-long)

Challenge:	Using Earley's context-free parsing algorithm, we show that the speed-ups are effective in practice.
Approach:	They propose a context-free parsing algorithm with various known and new speed-ups that improve Earley's (1970) O(N3\|G\|\|R\|) They also propose 'a binarized version' that achieves runtime of O(M\| \|G\| when the grammar is represented compactly as a single finite-state automaton M.
Outcome:	The proposed algorithm can be used to reduce the complexity of CKY on a binarized version of the grammar G.

Copied to clipboard

Fatemehsadat Mireshghallah, Yu Su, Tatsunori Hashimoto, Jason Eisner, Richard Shin