Papers by Dan Jurafsky

Prompt-and-Rerank: A Method for Zero-Shot and Few-Shot Arbitrary Textual Style Transfer with Small Language Models (2022.emnlp-main)

Challenge:	Existing models fail to adapt to unfamiliar speakers and language varieties . however, there are significant gaps in the adaptation of certain varieties based on the test speaker, variety, or recording conditions .
Approach:	They propose a framework that allows for in-context learning in Phi-4 Multimodal . they find that as few as 12 example utterances reduce word error rates by 19.7% .
Outcome:	The proposed framework reduces word error rates by 19.7% across diverse English corpora.

Copied to clipboard

Mirac Suzgun, Luke Melas-Kyriazi, Dan Jurafsky

Low-Resource NLP

Challenge:	a new method for textual style transfer is proposed for text with a limited set of style choices . textual styles are a complex task that requires specialized models to perform .
Approach:	They propose a method for arbitrary textual style transfer using pre-trained language models . they use a mathematical formulation of the TST task, decomposing it into three components .
Outcome:	The proposed method performs on par with state-of-the-art large-scale models while using less compute and memory.

Rethinking Word Similarity: Semantic Similarity through Classification Confusion (2025.naacl-long)

Copied to clipboard

Kaitlyn Zhou, Haishan Gao, Sarah Li Chen, Dan Edelstein, Dan Jurafsky, Chen Shani

Sharp Nearby, Fuzzy Far Away: How Neural Language Models Use Context (P18-1)

Challenge:	Word similarity measures cannot capture context-dependent, asymmetrical, polysemous nature of semantic similarity.
Approach:	They propose a new measure of similarity that reframes semantic similarity in terms of feature-based classification confusion.
Outcome:	The proposed model is comparable to cosine similarity in matching human similarity judgments across several datasets and can measure similarity using predetermined features of interest.

Copied to clipboard

Urvashi Khandelwal, He He, Peng Qi, Dan Jurafsky

Accommodation and Epistemic Vigilance: A Pragmatic Account of Why LLMs Fail to Challenge Harmful Beliefs (2026.acl-long)

Challenge:	Recent studies have shed light on the information encoded by long-term memory networks.
Approach:	They propose to use a neural caching model to model the role of context in an LSTM LM . they analyze the increase in perplexity when prior context words are shuffled, replaced, or dropped .
Outcome:	The proposed model is highly sensitive to the order of words within the most recent sentence, but ignores word order in the long-range context, suggesting the distant past is modeled only as a rough semantic field or topic.

Copied to clipboard

Myra Cheng, Robert D. Hawkins, Dan Jurafsky

RtGender: A Corpus for Studying Differential Responses to Gender (L18-1)

Challenge:	Recent studies show that large language models fail to challenge users’ harmful beliefs in domains ranging from medical advice to social reasoning.
Approach:	They propose to examine whether pragmatic factors influence LLM accommodation and epistemic vigilance in humans.
Outcome:	The proposed model can be understood and addressed as having excessive accommodation and insufficient epistemic vigilance.

Copied to clipboard

Rob Voigt, David Jurgens, Vinodkumar Prabhakaran, Dan Jurafsky, Yulia Tsvetkov

Challenge:	Prior work on linguistic gender difference and communications about gender has focused on language about or portraying persons of a particular gender.
Approach:	They present a multi-genre corpus of 25M comments from five socially and topically diverse sources tagged for the gender of the addressee and 30k annotations for sentiment and relevance of these responses.
Outcome:	The proposed dataset shows that responses to women are more emotive and about the speaker as an individual (rather than about the content being responded to).

When Do Pre-Training Biases Propagate to Downstream Tasks? A Case Study in Text Summarization (2023.eacl-main)

Copied to clipboard

Faisal Ladhak, Esin Durmus, Mirac Suzgun, Tianyi Zhang, Dan Jurafsky, Kathleen McKeown, Tatsunori Hashimoto

Summarization

Challenge:	Existing studies have shown that large language models contain linguistic and societal biases, but it is unclear how these biase amplify to downstream tasks.
Approach:	They investigate how name-nationality bias propagates from pre-training to downstream tasks . they show that these biases manifest themselves as hallucinations in summarization .
Outcome:	The proposed model can reduce the rate of hallucinations, but does not change the types of biases that do appear.

Improving Factual Completeness and Consistency of Image-to-Text Radiology Report Generation (2021.naacl-main)

Copied to clipboard

Yasuhide Miura, Yuhao Zhang, Emily Tsai, Curtis Langlotz, Dan Jurafsky

Multimodality

Challenge:	Existing report generation systems suffer from incomplete and inconsistent generation, despite achieving high performance on natural language metrics such as CIDEr and BLEU.
Approach:	They propose two new rewards that encourage the generation of factually complete and consistent radiology reports by using an existing semantic equivalence metric.
Outcome:	The proposed system significantly improves the F1 score of a clinical information extraction performance on two open radiology report datasets.

Making More of Little Data: Improving Low-Resource Automatic Speech Recognition Using Data Augmentation (2023.acl-long)

Copied to clipboard

Martijn Bartelds, Nay San, Bradley McDonnell, Dan Jurafsky, Martijn Wieling

Multimodality

Challenge:	Using self-training or text-to-speech (TTS) to improve low-resource ASR performance is costly and can lead to catastrophic forgetting.
Approach:	They examine whether data augmentation techniques could help improve low-resource ASR performance . they use self-training to generate transcriptions, which are combined with original data to train new system .
Outcome:	The proposed approach yields a 20.5% reduction in WER compared to a system trained on 24 minutes of manually transcribed speech.

Navigating the Grey Area: How Expressions of Uncertainty and Overconfidence Affect Language Models (2023.emnlp-main)

Copied to clipboard

Kaitlyn Zhou, Dan Jurafsky, Tatsunori Hashimoto

Framing and Agenda-setting in Russian News: a Computational Analysis of Intricate Political Strategies (D18-1)

Challenge:	a recent study examines how epistemic markers of certainty, uncertainty, or evidentiality affect LMs' attitudes toward knowledge . accuracies of expressions of high certainty and low certainty are associated with questions . prior work focused on learning the mapping between internal probabilities of a model and an ordinal output .
Approach:	They develop a typology of epistemic markers and inject 50 markers into questions . they find that LMs are highly sensitive to epistemical markers in prompts .
Outcome:	The proposed model epistemology study shows that LMs are sensitive to epistemic markers in prompts . expressions of high certainty result in 7% decrease in accuracy, while factive verbs hurt performance .

Copied to clipboard

Anjalie Field, Doron Kliger, Shuly Wintner, Jennifer Pan, Dan Jurafsky, Yulia Tsvetkov

Challenge:	Amidst growing concern over media manipulation, NLP studies focus on overt strategies like censorship and “fake news”.
Approach:	They propose to use two concepts from political science literature to identify subtler media manipulation strategies . they propose to apply embedding-based methods to cross-lingually project English frames to Russian .
Outcome:	The proposed techniques can be applied to 13 years of the Russian newspaper Izvestia and show that they highlight U.S. moral failings and threats to the U.s.

Learning Music Helps You Read: Using Transfer to Study Linguistic Structure in Language Models (2020.emnlp-main)

Copied to clipboard

Isabel Papadimitriou, Dan Jurafsky

Beyond Tokens: Concept-Level Training Objectives for LLMs (2026.eacl-short)

Challenge:	et al., 2018a, 2018b) show that LSTMs can transfer from non-linguistic data to natural language models with different types of abstract structure.
Approach:	They propose to use transfer learning to analyze encoding of grammatical structure in neural language models.
Outcome:	The proposed method improves test performance on natural language despite no overlap in surface form or vocabulary.

Copied to clipboard

Laya Iyer, Pranav Somani, Alice Guo, Dan Jurafsky, Chen Shani

Challenge:	Large language models (LLMs) are trained with a surprisingly narrow objective: predicting the next token in a sequence.
Approach:	They propose a shift from token-level to concept-level prediction where concepts group multiple surface forms of the same idea.
Outcome:	The proposed model improves on human-level models on diverse NLP benchmarks.

Integrating Text and Image: Determining Multimodal Document Intent in Instagram Posts (D19-1)

Copied to clipboard

Julia Kruk, Jonah Lubin, Karan Sikka, Xiao Lin, Dan Jurafsky, Ajay Divakaran

Multimodality

Challenge:	Existing studies on text-image content have focused on image as primary content, and text as secondary content.
Approach:	They propose a multimodal dataset of 1299 Instagram posts labeled for three orthogonal taxonomies . they show that employing both text and image improves intent detection by 9.6 .
Outcome:	The proposed model shows that using both text and image improves intent detection by 9.6 compared to using only the image modality.

Modular Domain Adaptation (2022.findings-acl)

Copied to clipboard

Junshen Chen, Dallas Card, Dan Jurafsky

AnthroScore: A Computational Linguistic Measure of Anthropomorphism (2024.eacl-long)

Challenge:	Existing models for sentiment analysis and hate speech detection are difficult to account for domain shift without access to source data.
Approach:	They propose to treat domain adaptation as a modular process that involves separate model producers and model consumers . they demonstrate that they can independently cooperate to facilitate more accurate measurements of text .
Outcome:	The proposed methods improve out-of-domain accuracy on four multi-domain text classification datasets.

Copied to clipboard

Myra Cheng, Kristina Gligoric, Tiziano Piccardi, Dan Jurafsky

Can Unconfident LLM Annotations Be Used for Confident Conclusions? (2025.naacl-long)

Challenge:	Anthropomorphism is commonplace in people's interactions with technology . anthropomorphizing language can suggest undue accountability and agency in technologies .
Approach:	They propose an automatic metric of implicit anthropomorphism in language . they use a masked language model to quantify how non-human entities are implicitly framed as human by the surrounding context.
Outcome:	The proposed metric measures how non-human entities are implicitly framed as human by the surrounding context.

Copied to clipboard

Kristina Gligoric, Tijana Zrnic, Cinoo Lee, Emmanuel Candes, Dan Jurafsky

Challenge:	Large language models (LLMs) have shown high agreement with human raters across a variety of tasks, demonstrating potential to ease the challenges of human data collection.
Approach:	They propose a method that combines LLM annotations and LLM confidence indicators to strategically select which human annotations to use.
Outcome:	The proposed method produces accurate estimates and valid confidence intervals while reducing the number of human annotations by over 25%.

JESC: Japanese-English Subtitle Corpus (L18-1)

Copied to clipboard

Reid Pryzant, Youngjoo Chung, Dan Jurafsky, Denny Britz

Machine Translation

Challenge:	Existing data on Japanese-English subtitles are limited due to the high cost of manual construction.
Approach:	They describe the Japanese-English Subtitle Corpus by crawling and aligning subtitles found on the web.
Outcome:	The JESC dataset covers the underrepresented domain of conversational dialogue.

Attention Flows are Shapley Value Explanations (2021.acl-short)

Copied to clipboard

Kawin Ethayarajh, Dan Jurafsky

Explainability & Interpretability in NLP

Challenge:	Shapley Values are a popular type of explanation in machine learning, but leave-one-out and attention-based explanations still predominate in NLP.
Approach:	They propose to use attention flow to explain the importance of features, embeddings, and even neurons to explain credit assignment problems in cooperative game theory.
Outcome:	The proposed explanations can explain the importance of features, embeddings, and even neurons, but in NLP, leave-one-out and attention-based explanations still predominate.

Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Models (2023.acl-long)

Copied to clipboard

Myra Cheng, Esin Durmus, Dan Jurafsky

Problems with Cosine as a Measure of Embedding Similarity for High Frequency Words (2022.acl-short)

Challenge:	Existing methods to measure stereotypes in large language models rely on manual templates or natural sentences that contain stereotypes.
Approach:	They propose a prompt-based method to measure stereotypes in large language models . they use natural language descriptions of the target demographic group alongside unmarked defaults .
Outcome:	The proposed method detects that portrayals contain higher rates of racial stereotypes than human-written portrayals.

Copied to clipboard

Kaitlyn Zhou, Kawin Ethayarajh, Dallas Card, Dan Jurafsky

Mini But Mighty: Efficient Multilingual Pretraining with Linguistically-Informed Data Selection (2023.findings-eacl)

Challenge:	We find that word similarities estimated by cosine over contextual embeddings are understated and trace this effect to training data frequency.
Approach:	They propose to use cosine similarity to estimate word similarities in contextual embeddings to trace this effect to training data frequency.
Outcome:	The proposed model underestimates similarity between frequent and low frequency words even after controlling for polysemy and other factors.

Copied to clipboard

Tolulope Ogunremi, Dan Jurafsky, Christopher Manning

Multilingual BERT has an accent: Evaluating English influences on fluency in multilingual models (2023.findings-eacl)

Challenge:	AfriBERTa shows that training transformer models from scratch on 1GB of data from many unrelated African languages outperforms massively multilingual models on downstream NLP tasks.
Approach:	They propose that training on smaller amounts of data but from related languages could match the performance of models trained on large, unrelated data.
Outcome:	The proposed model outperforms models trained on large, unrelated datasets on downstream NLP tasks.

Copied to clipboard

Isabel Papadimitriou, Kezia Lopez, Dan Jurafsky

With Little Power Comes Great Responsibility (2020.emnlp-main)

Challenge:	Multilingual models can improve NLP performance on low-resource languages by leveraging higher-resourced languages, but they also reduce average performance on all languages.
Approach:	They propose a method to evaluate multilingual models by asking if models predict languages with an 'English accent' they propose to use grammatical structure bias to determine if multilingual model is biased toward English-like setting .
Outcome:	The proposed method compares the fluency of multilingual models to the fluencies of monolingual Spanish and Greek models.

Copied to clipboard

Dallas Card, Peter Henderson, Urvashi Khandelwal, Robin Jia, Kyle Mahowald, Dan Jurafsky

Utility is in the Eye of the User: A Critique of NLP Leaderboards (2020.emnlp-main)

Challenge:	Underpowered experiments make it more difficult to discern the difference between statistical noise and meaningful model improvements and increase the chances of exaggerated findings.
Approach:	They characterize typical statistical power for a variety of settings and characterize it by a set of existing NLP papers and datasets.
Outcome:	The authors characterize typical power for a variety of settings and find it common in the literature.

Copied to clipboard

Kawin Ethayarajh, Dan Jurafsky

Green & Sustainable NLP

Challenge:	Performance-based evaluation has been at the expense of other attributes valued by the NLP community, such as compactness and energy efficiency.
Approach:	They propose to frame both the leaderboard and NLP practitioners as consumers and the benefit they get from a model as its utility to them.
Outcome:	The proposed model size and energy efficiency benchmarks have been successful in driving the creation of more accurate models, but have been at the expense of other attributes valued by the NLP community.

🧑‍🍳 Cooking Up Creativity: Enhancing LLM Creativity through Structured Recombination (2026.tacl-1)

Copied to clipboard

Moran Mizrahi, Chen Shani, Gabriel Stanovsky, Dan Jurafsky, Dafna Shahaf

Challenge:	Large Language Models excel at many tasks, yet struggle to generate truly creative ideas.
Approach:	They propose a novel approach that enhances Large Language Models' creativity by manipulating structured representations of existing ideas.
Outcome:	The proposed model outperforms GPT-4o in novelty and diversity and outperformed GPT-0 in creative generation.

Richer Countries and Richer Representations (2022.findings-acl)

Copied to clipboard

Kaitlyn Zhou, Kawin Ethayarajh, Dan Jurafsky

Representation Learning

Challenge:	Using BERT, countries with low frequency in training data are less likely to be invocabulary, and are less frequently predicted in the masked language modeling task.
Approach:	They propose three criteria to characterize the quality of representations for particular entities or groups: consistency, distinctiveness, and recognizability.
Outcome:	The results suggest that frequency is highly correlated with a country’s GDP, perpetuating historic power and wealth inequalities.

Detecting Stance in Media On Global Warming (2020.findings-emnlp)

Copied to clipboard

Yiwei Luo, Dallas Card, Dan Jurafsky

Challenge:	a global warming denier might frame an opinion of an untrustworthy source with a predicate connoting doubt.
Approach:	They propose a dataset of stance-labeled GW sentences and train a BERT classifier to study opinion-framing in the global warming debate.
Outcome:	The proposed dataset of stance-labeled GW sentences and a BERT classifier study opinion-framing in the global warming debate.

Recursive Routing Networks: Learning to Compose Modules for Language Understanding (N19-1)

Copied to clipboard

Ignacio Cases, Clemens Rosenbaum, Matthew Riemer, Atticus Geiger, Tim Klinger, Alex Tamkin, Olivia Li, Sandhini Agarwal, Joshua D. Greene, Dan Jurafsky, Christopher Potts, Lauri Karttunen

Challenge:	Recursive Routing Networks are modular, adaptable models that learn effectively in diverse environments.
Approach:	They propose to apply Recursive Routing Networks (RRNs) to natural language understanding by integrating them into existing architectures and recurrent network hidden layers.
Outcome:	The proposed model optimizes the parameters of the functions and the meta-learner decision-making component for routing inputs through those functions.

HumT DumT: Measuring and controlling human-like language in LLMs (2025.acl-long)

Copied to clipboard

Myra Cheng, Sunny Yu, Dan Jurafsky

Challenge:	Human-like language might improve user experience, but might also lead to deception, overreliance, and stereotyping.
Approach:	They introduce HumT and SocioT, metrics for human-like tone in LLM outputs . HumT measures human-type tone and other dimensions of social perceptions in text data .
Outcome:	The proposed method reduces human-like tone while preserving model performance.

Injecting structural hints: Using language models to study inductive biases in language learning (2023.findings-emnlp)

Copied to clipboard

Isabel Papadimitriou, Dan Jurafsky

Pretraining with Contrastive Sentence Objectives Improves Discourse Performance of Language Models (2020.acl-main)

Challenge:	a recent study examines the cognitive inductive biases that make language learning possible.
Approach:	They structurally bias transformer language models by pretraining on synthetic data . they then evaluate their inductive biases by fine-tuning on three different languages .
Outcome:	The proposed method predisposes transformer models to three types of inductive biases . it also fine-tunes the models on three typologically-distant human languages .

Copied to clipboard

Dan Iter, Kelvin Guu, Larry Lansing, Dan Jurafsky

False Friends Are Not Foes: Investigating Vocabulary Overlap in Multilingual Language Models (2025.findings-emnlp)

Challenge:	Recent models for unsupervised representation learning of text have put little focus on discourse-level representations.
Approach:	They propose an inter-sentence objective for pretraining language models that models discourse coherence and the distance between sentences.
Outcome:	The proposed model outperforms the BERT-Large model on the discourse representation benchmark DiscoEval and yields gains of 2%-6% absolute even for tasks that do not explicitly evaluate discourse.

Copied to clipboard

Julie Kallini, Dan Jurafsky, Christopher Potts, Martijn Bartelds

Multilinguality

Challenge:	Prior work has shown that token overlap facilitates cross-lingual transfer or introduces interference between languages?
Approach:	They devised a controlled experiment where they train bilingual autoregressive models on multiple language pairs under systematically varied vocabulary overlap settings.
Outcome:	The proposed model outperforms models with disjointed vocabularies on XNLI and XQuAD and shows that token overlap is beneficial for multilingual tokenizers.

Social Bias Frames: Reasoning about Social and Power Implications of Language (2020.acl-main)

Copied to clipboard

Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Jurafsky, Noah A. Smith, Yejin Choi

The Authenticity Gap in Human Evaluation (2022.emnlp-main)

Challenge:	Language has enormous power to project social biases and reinforce stereotypes on people.
Approach:	They propose a new conceptual formalism that aims to model the pragmatic frames in which people project social biases and power differentials onto others.
Outcome:	The proposed model can model the pragmatic frames in which people project social biases and power differentials onto others.

Copied to clipboard

Kawin Ethayarajh, Dan Jurafsky

Text Generation

Challenge:	Using the standard protocol to evaluate NLGs is often violated, resulting in annotator ratings cease to reflect their preferences.
Approach:	They propose a human evaluation protocol called system-level probabilistic assessment (SPA) this protocol is based on the assumption that annotators are biased by likert scales .
Outcome:	The proposed protocol can recover the ordering of GPT-3 models by size, but less than half of the expected preferences can be recovered when human evaluation is done with the standard protocol.

Causal Effects of Linguistic Properties (2021.naacl-main)

Copied to clipboard

Reid Pryzant, Dallas Card, Dan Jurafsky, Victor Veitch, Dhanya Sridhar

Challenge:	Social scientists have long been interested in the causal effects of language, studying questions like: How should political candidates describe their personal history to appeal to voters?
Approach:	They propose an algorithm for estimating causal effects of linguistic properties that leverages distant supervision and a pre-trained language model to adjust for the text.
Outcome:	The proposed method outperforms other methods when estimating the effect of Amazon review sentiment on semi-simulated sales figures.

Measuring Conversational Uptake: A Case Study on Student-Teacher Interactions (2021.acl-long)

Copied to clipboard

Dorottya Demszky, Jing Liu, Zid Mancenido, Julie Cohen, Heather Hill, Dan Jurafsky, Tatsunori Hashimoto

Natural Language Interfaces

Challenge:	Despite extensive research showing the positive impact of uptake on student learning and achievement, there is little evidence that it is effective in teaching.
Approach:	They propose a framework for computationally measuring uptake by releasing a dataset of student-teacher exchanges extracted from US math classroom transcripts annotated for uptake . they formalize uptake as pointwise Jensen-Shannon Divergence (pJSD) and conduct a linguistically-motivated comparison of different unsupervised measures.
Outcome:	The proposed framework outperforms baseline measures in identifying uptake phenomena like question answering and reformulation.

Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory (2026.eacl-long)

Copied to clipboard

Mirac Suzgun, Mert Yuksekgonul, Federico Bianchi, Dan Jurafsky, James Zou

string2string: A Modern Python Library for String-to-String Algorithms (2024.acl-demos)

Challenge:	Unlike fine-tuning or static retrieval methods, DC adapts LMs’ problem-solving skills on the fly, without modifying their underlying parameters.
Approach:	They propose a lightweight framework that endows a black-box LM with a persistent, evolving memory.
Outcome:	The proposed framework enables models to store and reuse accumulated strategies, code snippets, and general problem-solving insights at inference time.

Copied to clipboard

Mirac Suzgun, Stuart Shieber, Dan Jurafsky

Noising and Denoising Natural Language: Diverse Backtranslation for Grammar Correction (N18-1)

Challenge:	Notable algorithms include the Smith-Waterman algorithm for pairwise local alignment, the Hirschberg algorithm for global alignment, and the Wagner-Fischer algorithm for edit distance.
Approach:	string2string is an open-source library that offers efficient algorithms for string-to-string problems.
Outcome:	string2string is an open-source library that offers efficient algorithms for string-to-string problems.

Copied to clipboard

Ziang Xie, Guillaume Genthial, Stanley Xie, Andrew Ng, Dan Jurafsky

Machine Translation

Challenge:	Existing grammar correction methods are limited in their ability to correct highly local errors . evaluators are unable to distinguish noisy examples from nonsynthesized ones .
Approach:	They propose to synthesize parallel data by noising a clean monolingual corpus . they propose to apply noise to the corpus to syntherize additional noisy examples .
Outcome:	The proposed method can produce almost as strong results as training with nonsynthesized data.

Let’s Make Your Request More Persuasive: Modeling Persuasive Strategies via Semi-Supervised Neural Nets on Crowdfunding Platforms (N19-1)

Copied to clipboard

Diyi Yang, Jiaao Chen, Zichao Yang, Dan Jurafsky, Eduard Hovy

Textual Analogy Parsing: What’s Shared and What’s Compared among Analogous Facts (D18-1)

Challenge:	Existing models can't quantify persuasiveness of requests or extract successful persuasive strategies.
Approach:	They propose a semi-supervised hierarchical neural network model to quantify persuasiveness and identify persuasive strategies in advocacy requests.
Outcome:	The proposed method outperforms baseline models and offers increased interpretability of persuasive speech.

Copied to clipboard

Matthew Lamm, Arun Chaganty, Christopher D. Manning, Dan Jurafsky, Percy Liang

Reasoning

Challenge:	Existing methods to extract information from text do not capture disparity between demographic groups.
Approach:	They propose a task of Textual Analogy Parsing to model higher-order meanings by comparing poverty rates between different demographic groups.
Outcome:	The proposed model can be used to generate graphs from quantitative text.

REL-A.I.: An Interaction-Centered Approach To Measuring Human-LM Reliance (2025.naacl-long)

Copied to clipboard

Kaitlyn Zhou, Jena D. Hwang, Xiang Ren, Nouha Dziri, Dan Jurafsky, Maarten Sap

Attention to Non-Adopters (2026.findings-acl)

Challenge:	Existing evaluations of large language models' ability to communicate uncertainty and knowledge limitations focus on the behaviors of their human interlocutors.
Approach:	They propose an interaction-centered evaluation approach that quantifies whether and how humans rely on LLMs' responses.
Outcome:	The proposed approach quantifies whether and how humans rely on LLMs' responses.

Copied to clipboard

Kaitlyn Zhou, Kristina Gligorić, Myra Cheng, Michelle S. Lam, Vyoma Raman, Boluwatife Aminu, Caeley Woo, Michael Brockman, Hannah Cha, Dan Jurafsky

Deconfounded Lexicon Induction for Interpretable Social Science (N18-1)

Challenge:	incorporating non-adopter perspectives is essential for developing useful and capable LLMs, argues a new study.
Approach:	They argue that incorporating non-adopter perspectives is essential for developing broadly useful and capable LLMs.
Outcome:	The proposed method will risk missing tasks prioritized by non-adopters, the authors argue . they show that non-dots diverge from those of current users, and non-no-acopter needs point towards novel reasoning tasks.

Copied to clipboard

Reid Pryzant, Kelly Shen, Dan Jurafsky, Stefan Wagner

Focus on what matters: Applying Discourse Coherence Theory to Cross Document Coreference (2021.emnlp-main)

Challenge:	Lexical features are useful beyond predictive performance. they can also be used to understand the subjective properties of a text.
Approach:	They propose two deep learning algorithms that separate the explanatory power of text from confounds.
Outcome:	The proposed algorithms are predictive of a set of target variables yet uncorrelated to confounds . they pick words associated with narrative persuasion and are more predictive than standard features .

Copied to clipboard

William Held, Dan Iter, Dan Jurafsky

Information Extraction & Text Mining

Challenge:	Existing approaches simplify by considering coreference only within document clusters, but this fails to handle inter-cluster coreference, common in many applications.
Approach:	They propose to model entities/events in a reader’s focus as a neighborhood within a learned latent embedding space which minimizes the distance between mentions and the centroids of their gold coreference clusters.
Outcome:	The proposed model achieves state-of-the-art for events and entities on the ECB+, Gun Violence, Football Coreference, and Cross-Domain Cross-DDocument Coreference corpora.

Sensitivity as a Complexity Measure for Sequence Classification Tasks (2021.tacl-1)

Copied to clipboard

Michael Hahn, Dan Jurafsky, Richard Futrell

Information Extraction & Text Mining

Challenge:	Existing complexity metrics provide limited practical insight into complexity differences between tasks.
Approach:	They propose a theoretical framework for understanding and predicting the complexity of sequence classification tasks using a new extension of the theory of Boolean function sensitivity.
Outcome:	The proposed framework predicts the complexity of sequence classification tasks using a new method . it shows that low-sensitivity functions are easier to learn for LSTMs than lexical classifiers .

Grounding Gaps in Language Model Generations (2024.naacl-long)

Copied to clipboard

Omar Shaikh, Kristina Gligoric, Ashna Khetan, Matthias Gerstgrasser, Diyi Yang, Dan Jurafsky

Dialogue Systems & Conversational Agents

Challenge:	Effective conversation requires common ground, but it does not emerge spontaneously.
Approach:	They propose a set of grounding acts and metrics that quantify attempted grounding . they find that large language models generate language with less conversational grounding than humans .
Outcome:	The proposed models generate language with less conversational grounding than humans . compared to humans, they generate language that appears to presume common ground .

NLP Systems That Can’t Tell Use from Mention Censor Counterspeech, but Teaching the Distinction Helps (2024.naacl-long)

Copied to clipboard

Kristina Gligoric, Myra Cheng, Lucia Zheng, Esin Durmus, Dan Jurafsky