Papers by Swabha Swayamdipta
Investigating the Benefits of Free-Form Rationales (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that crowdsourced rationales provide additional background knowledge to models . a qualitative study shows generated rationale is not as useful for humans as crowdsourced ones . |
| Approach: | They investigate whether crowdsourced rationales provide additional background knowledge to models . they find that ECQA rationale provides additional background information to understand a decision . |
| Outcome: | The results show that ECQA rationales provide additional background knowledge to understand a decision . compared to crowdsourced rationale, generated rationale is not as useful for humans . |
Polyglot Semantic Role Labeling (P18-2)
Copied to clipboard
| Challenge: | Existing approaches to multilingual semantic dependency parsing treat languages independently, without exploiting similarities between semantic structures across languages. |
| Approach: | They propose to combine resources from different languages in a CoNLL 2009 shared task to build a single polyglot semantic dependency parser. |
| Outcome: | The proposed model outperforms monolingual training on a CoNLL 2009 dataset with training data from multiple languages and representations using multilingual word vectors. |
Contrastive Explanations for Model Interpretability (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for producing model explanations seek all causal factors at once, making them difficult to comprehend. |
| Approach: | They propose a method to produce contrastive explanations in the latent space . they use attribution and token/span attribution to produce models that consider only contrastive reasoning . |
| Outcome: | The proposed method allows model behavior to consider only contrastive reasoning . it also uncovers which aspects of the input are useful for and against particular decisions . |
COBRA Frames: Contextual Reasoning about Effects and Harms of Offensive Statements (2023.findings-acl)
Copied to clipboard
Xuhui Zhou, Hao Zhu, Akhila Yerukola, Thomas Davidson, Jena D. Hwang, Swabha Swayamdipta, Maarten Sap
| Challenge: | Understanding the harms and offensiveness of statements requires reasoning about the social and situational context in which they are made. |
| Approach: | They propose a context-aware formalism for explaining the intents, reactions, and harms of offensive statements grounded in their social and situational contexts. |
| Outcome: | The proposed framework is the first context-aware formalism for explaining the intents, reactions, and harms of offensive statements grounded in their social and situational context. |
Learning Joint Semantic Parsers from Disjoint Data (N18-1)
Copied to clipboard
| Challenge: | Various formal meaning representations have been developed corresponding to different semantic theories. |
| Approach: | They propose a method to learn a semantic parser from multiple datasets by treating annotations for unobserved formalisms as latent structured variables. |
| Outcome: | The proposed approach improves on existing methods using unobserved formalisms and underlying corpora. |
Transfer Learning in Natural Language Processing (N19-5)
Copied to clipboard
| Challenge: | supervised machine learning is based on learning in isolation, a single predictive model for a task using a dataset. |
| Approach: | They present an overview of modern transfer learning methods in natural language processing . they review examples and case studies on how models can be integrated and adapted . |
| Outcome: | The proposed methods improve upon the state-of-the-art on a wide range of NLP tasks. |
Syntactic Scaffolds for Semantic Structures (D18-1)
Copied to clipboard
| Challenge: | Syntactic scaffolds avoid expensive syntactical processing at runtime . many systems have used syntastic dependency or phrase-based parsers as preprocessing for semantic analysis. |
| Approach: | They propose a multitask learning approach that uses a syntactic treebank to integrate syntaktic information into semantic tasks. |
| Outcome: | The proposed method improves on PropBank semantics, frame semantics and coreference resolution tasks. |
Generative Data Augmentation for Commonsense Reasoning (2020.findings-emnlp)
Copied to clipboard
Yiben Yang, Chaitanya Malaviya, Jared Fernandez, Swabha Swayamdipta, Ronan Le Bras, Ji-Ping Wang, Chandra Bhagavatula, Yejin Choi, Doug Downey
| Challenge: | Recent advances in commonsense reasoning depend on large-scale human-authored training data. |
| Approach: | They propose a generative data augmentation technique that augments human-authored training data by using pretrained language models. |
| Outcome: | The proposed technique outperforms existing methods on commonsense reasoning benchmarks and enhances out-of-distribution generalization. |
WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing datasets are often flooded with repetitive and spurious patterns, leading to a lack of linguistic diversity. |
| Approach: | They propose a method that uses cartography to automatically identify and filter examples that demonstrate challenging reasoning patterns and then automatically compose new ones with similar patterns. |
| Outcome: | The proposed approach improves performance on eight out-of-domain test sets compared to training on the 4x larger MultiNLI dataset. |
DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts (2021.acl-long)
Copied to clipboard
Alisa Liu, Maarten Sap, Ximing Lu, Swabha Swayamdipta, Chandra Bhagavatula, Noah A. Smith, Yejin Choi
| Challenge: | Decoding-time Experts is a decoding- time method for controlled text generation . it combines a pretrained language model with "expert" LMs and/or "anti-expert" experts . |
| Approach: | They propose a decoding-time method that combines a pretrained language model with "expert" LMs and/or "anti-expert" experts to generate controlled text. |
| Outcome: | The proposed method outperforms existing controllable generation methods on automatic and human evaluations. |
The Right Tool for the Job: Matching Model and Instance Complexities (2020.acl-main)
Copied to clipboard
| Challenge: | a large increase in the size of NLP models can increase production costs and reduce adoption on real-time devices. |
| Approach: | They propose a modification to contextual representation fine-tuning which allows for an early exit from neural network calculations for simple instances and late exit for hard instances. |
| Outcome: | The proposed method produces models which are up to five times faster than the state of the art while preserving their accuracy. |
Annotation Artifacts in Natural Language Inference Data (N18-2)
Copied to clipboard
| Challenge: | Large-scale datasets for natural language inference are created by crowdsourcing annotations . authors show that success of natural language models to date has been overestimated . |
| Approach: | They propose a method for crowdsourcing annotations to generate 3 new sentences based on a sentence (premise) they show that a simple text categorization model can correctly classify the hypothesis alone in about 67% of SNLI and 53% of MultiNLI . |
| Outcome: | The proposed model can classify the hypothesis alone in 67% of SNLI and 53% of MultiNLI datasets. |
Challenges in Automated Debiasing for Toxic Language Detection (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for debiasing toxic language data are limited in their ability to prevent biased behavior in toxic language detection systems. |
| Approach: | They propose to debiase toxic language detection models using lexical and dialectal markers using synthetic labels instead of traditional methods. |
| Outcome: | The proposed method reduces dialectal associations with toxicity despite the use of synthetic labels . |
NeuroComparatives: Neuro-Symbolic Distillation of Comparative Knowledge (2024.findings-naacl)
Copied to clipboard
| Challenge: | Comparative knowledge is an essential component of our world knowledge, yet understudied in prior literature. |
| Approach: | They propose a framework for comparative knowledge distillation overgenerated from language models . they use a corpus of 8.8M comparisons over 1.74M entity pairs to acquire comparative information . |
| Outcome: | The proposed framework acquires comparative knowledge between everyday objects . human evaluations show that it outperforms existing resources in terms of validity . |
NeuroCounterfactuals: Beyond Minimal-Edit Counterfactuals for Richer Data Augmentation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to produce counterfactuals rely on small perturbations via minimal edits, resulting in simplistic changes. |
| Approach: | They propose a novel approach to produce counterfactuals that allow for larger edits and linguistic diversity while still bearing similarity to the original document. |
| Outcome: | The proposed approach outperforms existing methods for generalizing natural language models under select settings. |
Believing without Seeing: Quality Scores for Contextualizing Vision-Language Model Explanations (2026.acl-long)
Copied to clipboard
| Challenge: | Prior work has found that explanations can easily convince users that inaccurate VLM predictions are correct. |
| Approach: | They propose to evaluate two complementary qualities of VLM-generated explanations via two quality scoring functions to improve their accuracy. |
| Outcome: | The proposed explanations improve accuracy on the A-OKVQA, VizWiz, and MMMU-Pro tasks by 11.1%, including a 15.4% reduction in falsely believing incorrect predictions. |
I2D2: Inductive Knowledge Distillation with NeuroLogic and Self-Imitation (2023.acl-long)
Copied to clipboard
Chandra Bhagavatula, Jena D. Hwang, Doug Downey, Ronan Le Bras, Ximing Lu, Lianhui Qin, Keisuke Sakaguchi, Swabha Swayamdipta, Peter West, Yejin Choi
| Challenge: | Empirical results suggest that scale is not the only way to build commonsense capabilities. |
| Approach: | They propose a commonsense distillation framework that can achieve a competitive level of commonsensing without relying on the benefits of scale. |
| Outcome: | The proposed framework breaks the dependence on the extreme-scale teacher model with two innovations: (1) the novel adaptation of NeuroLogic Decoding to enhance the generation quality of the weak, off-the-shelf language models, and (2) self-imitation learning to iteratively learn from the model’s own enhanced commonsense acquisition capabilities. |
Compare without Despair: Reliable Preference Evaluation with Generation Separability (2024.findings-emnlp)
Copied to clipboard
| Challenge: | a meta-evaluation measure, separability, estimates how suitable a test instance is for pairwise preference evaluation. |
| Approach: | They propose a measure of separability which measures how suitable a test instance is for pairwise preference evaluation. |
| Outcome: | The proposed measure shows that instances with high separability yield more consistent preference ratings from human- and auto-raters. |
Uncovering Intervention Opportunities for Suicide Prevention with Language Model Assistants (2026.acl-long)
Copied to clipboard
Jaspreet Ranjit, Hyundong Justin Cho, Claire J. Smerdon, Yoonsoo Nam, Myles Phung, Jonathan May, John R. Blosnich, Swabha Swayamdipta
| Challenge: | Using language models, annotators can help develop novel suicide interventions . 85% of cases where LM predictions disagree with existing annotations are analyzed . |
| Approach: | They propose a human-in-the-loop algorithm that leverages language models as an assistant to annotators and experts to facilitate data-driven insights from NVDRS data. |
| Outcome: | The proposed algorithm can be used to support the development of novel suicide interventions . it finds that LM predictions match existing data annotations about 85% of the time . |
Out-of-Distribution Detection through Soft Clustering with Non-Negative Kernel Regression (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for detecting out-of-distribution data are computationally complex and storage-intensive. |
| Approach: | They propose a soft clustering approach for OOD detection based on non-negative kernel regression . their approach greatly reduces computational and space complexities while retaining competitive performance. |
| Outcome: | The proposed approach outperforms existing methods by up to 4 AUROC points on four benchmarks while retaining competitive performance. |
ELI-Why: Evaluating the Pedagogical Utility of Language Model Explanations (2025.findings-acl)
Copied to clipboard
Brihi Joshi, Keyu He, Sahana Ramnath, Sadra Sabouri, Kaitlyn Zhou, Souti Chattopadhyay, Swabha Swayamdipta, Xiang Ren
| Challenge: | Language models are widely used in education, yet their ability to tailor responses to learners with varied informational needs and knowledge backgrounds remains under-explored. |
| Approach: | They conduct two extensive human studies to assess the utility of language model-generated explanatory answers (explanations) on a benchmark of 13.4K "Why" questions. |
| Outcome: | The proposed model explanations match learners' educational backgrounds only 50% of the time, compared to 79% for lay explanations. |
Evaluating Evaluation Metrics – The Mirage of Hallucination Detection (2025.findings-emnlp)
Copied to clipboard
Atharva Kulkarni, Yuan Zhang, Joel Ruben Antony Moniz, Xiou Ge, Bo-Hsiang Tseng, Dhivya Piraviperumal, Swabha Swayamdipta, Hong Yu
| Challenge: | a large-scale empirical evaluation of hallucination detection metrics is conducted . hallucinosity is a significant obstacle to the reliability and widespread adoption of language models . |
| Approach: | They conduct large-scale empirical evaluation of hallucination detection metrics . they compare hallucinian language models, language models and decoding methods . |
| Outcome: | The results show that the evaluations of hallucination detection metrics fail to align with human judgments, they say . they also show that evaluations with LLM-based evaluation yield the best overall results . |
Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks (2020.acl-main)
Copied to clipboard
Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, Noah A. Smith
| Challenge: | Language models prerained on text from a wide variety of sources form the foundation of today’s NLP. |
| Approach: | They propose to tailor a pretrained model to the domain of a target task by using domain-adaptive pretraining in-domain. |
| Outcome: | The proposed model can be tailored to the domain of a target task and perform well under both high- and low-resource settings. |
REV: Information-Theoretic Evaluation of Free-Text Rationales (2023.acl-long)
Copied to clipboard
| Challenge: | Existing metrics for rationale evaluation focus on the association between the rationale and a label, whereas REV is more sensitive to new information in free-text rationales. |
| Approach: | They propose a metric called REV to quantify the amount of new, label-relevant information in a rationale beyond the information already available in the input or the label. |
| Outcome: | The proposed metric is consistent with human judgments on rationale evaluations and provides more sensitive measurements of new information in free-text rationales. |
Frame Semantics across Languages: Towards a Multilingual FrameNet (C18-3)
Copied to clipboard
| Challenge: | This workshop will present current research on aligning Frame Semantic resources across languages . resources based on FrameNet have been created for roughly a dozen languages based upon Fillmore's Frame Sementics . |
| Approach: | This workshop will present current research on aligning Frame Semantic resources across languages . resources based on FrameNet have been created for roughly a dozen languages based upon Fillmore's Frame Sementics . |
| Outcome: | This workshop will present current research on aligning Frame Semantic resources across languages and automatic frame semantic parsing in English and other languages. |
Annotating FrameNet via Structure-Conditioned Language Generation (2024.acl-short)
Copied to clipboard
| Challenge: | despite the remarkable generative capabilities of language models, their effectiveness on explicit manipulation and generation of linguistic structures remains understudied. |
| Approach: | They propose a framework to generate frame-semantically annotated sentences following FrameNet . they use explicit semantic information to generate frames with high human acceptance . |
| Outcome: | The proposed framework produces frame-semantic annotations with high human acceptance . generating high-quality, semantically rich data is effective in low-resource settings, but not under higher resource settings. |
Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics (2020.emnlp-main)
Copied to clipboard
Swabha Swayamdipta, Roy Schwartz, Nicholas Lourie, Yizhong Wang, Hannaneh Hajishirzi, Noah A. Smith, Yejin Choi
| Challenge: | Large datasets have become commonplace in NLP research, but the emphasis on quantity has made it challenging to assess the quality of data. |
| Approach: | They propose a model-based tool to characterize and diagnose large datasets . they leverage the behavior of the model on individual instances during training . |
| Outcome: | Experiments on four datasets show that the tool can characterize and diagnose datasets with a model-based tool. |
Robust Data Watermarking in Language Models by Injecting Fictitious Knowledge (2025.findings-acl)
Copied to clipboard
| Challenge: | Data watermarking in language models injects traceable signals, such as specific token sequences or stylistic patterns, into copyrighted text, allowing copyright holders to track and verify training data ownership. |
| Approach: | They propose a data watermarking approach that injects coherent and plausible yet fictitious knowledge into training data using generated passages describing a fictious entity and its associated attributes. |
| Outcome: | The proposed method is designed to be memorized by the LLM, and that increasing their density, length, and diversity of attributes strengthens their memorization. |
Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection (2022.naacl-main)
Copied to clipboard
| Challenge: | toxicity annotations are often ignored because of its subjective nature and lack of nuance. |
| Approach: | They examine the effect of annotator identities and beliefs on toxic language annotations by considering posts with three characteristics: anti-Black language, African American English (AAE) dialect, and vulgarity. |
| Outcome: | The findings show strong associations between annotator identity and beliefs and ratings of toxicity. |
We’re Afraid Language Models Aren’t Modeling Ambiguity (2023.emnlp-main)
Copied to clipboard
Alisa Liu, Zhaofeng Wu, Julian Michael, Alane Suhr, Peter West, Alexander Koller, Swabha Swayamdipta, Noah Smith, Yejin Choi
| Challenge: | Ambiguity is an intrinsic feature of natural language, allowing us to anticipate misunderstandings and revise our interpretations as listeners. |
| Approach: | They use AmbiEnt to capture ambiguity in a sentence and analyze it to evaluate pretrained LMs. |
| Outcome: | The proposed model can flag political claims in the wild that are misleading due to ambiguity. |
Improving Language Model Personas via Rationalization with Psychological Scaffolds (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to building personas rely on a user’s demographic attributes and/or prior judgments, but not on any underlying reasoning behind a person’s judgments. |
| Approach: | They propose a framework that integrates rationales for why a user could have made a certain judgment into LM personas by incorporating potential rationale. |
| Outcome: | The proposed framework outperforms models conditioned on demographic attributes and/or prior judgments on public opinion and movie preference prediction tasks. |
Reframing Human-AI Collaboration for Generating Free-Text Explanations (2022.naacl-main)
Copied to clipboard
| Challenge: | Large language models are capable of generating fluent-appearing text with little task-specific supervision. |
| Approach: | They propose a pipeline that combines GPT-3 with a supervised filter that incorporates binary acceptability judgments from humans in the loop. |
| Outcome: | The proposed model can generate freetext explanations in a fewshot setting with human-written examples. |
OATH-Frames: Characterizing Online Attitudes Towards Homelessness with LLM Assistants (2024.emnlp-main)
Copied to clipboard
Jaspreet Ranjit, Brihi Joshi, Rebecca Dorn, Laura Petry, Olga Koumoundouros, Jayne Bottarini, Peichen Liu, Eric Rice, Swabha Swayamdipta
| Challenge: | a large-scale analysis of millions of tweets on homelessness is challenging to understand at scale. |
| Approach: | They propose a framing typology: Online Attitudes Towards Homelessness (OATH) They use large language models to analyze millions of tweets to find patterns in public attitudes . |
| Outcome: | The proposed model speeds up annotations while incurring a 3 point performance reduction compared to existing classifiers . |