Papers by Chandra Bhagavatula
Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning (D19-1)
Copied to clipboard
| Challenge: | Existing reading comprehension datasets focus on factual and literal understanding of context paragraphs, but our dataset focuses on reading between the lines over a diverse collection of everyday narratives. |
| Approach: | They propose a large-scale dataset that requires commonsense-based reading comprehension, formulated as multiple-choice questions. |
| Outcome: | The proposed architecture improves over the baselines of existing reading comprehension datasets and shows a significant gap between machine (68.4%) and human performance (94%). |
Thinking Like a Skeptic: Defeasible Inference in Natural Language (2020.findings-emnlp)
Copied to clipboard
Rachel Rudinger, Vered Shwartz, Jena D. Hwang, Chandra Bhagavatula, Maxwell Forbes, Ronan Le Bras, Noah A. Smith, Yejin Choi
| Challenge: | Defeasible inference is a mode of reasoning in which an inference may be weakened or overturned in light of new evidence. |
| Approach: | They propose a dataset for defeasible inference in natural language that includes extensions to existing inference datasets. |
| Outcome: | Defeasible NLI extends existing datasets for defeaasibility inference in natural language . generative models can weaken or strengthen inferences up to 68% of the time, it shows . |
proScript: Partially Ordered Scripts Generation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Scripts represent structured commonsense knowledge about prototypical events in everyday situations/scenarios such as bake a cake. |
| Approach: | They collect 6.4k crowdsourced partially ordered scripts and develop models that combine language generation and graph structure prediction to generate scripts. |
| Outcome: | The proposed models perform well on two tasks: edge prediction and script generation. |
NeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic Constraints (2021.naacl-main)
Copied to clipboard
| Challenge: | Conditional text generation often requires lexical constraints, i.e., which words should or shouldn't be included in the output text. |
| Approach: | They propose an algorithm that enables neural language models to generate fluent text while satisfying complex lexical constraints. |
| Outcome: | The proposed algorithm outperforms existing methods on four benchmarks and shows that it handles any set of lexical constraints expressible under predicate logic while its asymptotic runtime is equivalent to conventional beam search. |
Unsupervised Commonsense Question Answering with Self-Talk (2020.emnlp-main)
Copied to clipboard
| Challenge: | Current systems rely on pre-trained language models or external knowledge bases to incorporate additional relevant knowledge. |
| Approach: | They propose an unsupervised framework based on self-talk to improve commonsense performance by asking language models to ask information seeking questions. |
| Outcome: | Empirical results show that the proposed framework improves on four out of six commonsense benchmarks and competes with models that obtain knowledge from external KBs. |
Generative Data Augmentation for Commonsense Reasoning (2020.findings-emnlp)
Copied to clipboard
Yiben Yang, Chaitanya Malaviya, Jared Fernandez, Swabha Swayamdipta, Ronan Le Bras, Ji-Ping Wang, Chandra Bhagavatula, Yejin Choi, Doug Downey
| Challenge: | Recent advances in commonsense reasoning depend on large-scale human-authored training data. |
| Approach: | They propose a generative data augmentation technique that augments human-authored training data by using pretrained language models. |
| Outcome: | The proposed technique outperforms existing methods on commonsense reasoning benchmarks and enhances out-of-distribution generalization. |
“I’m Not Mad”: Commonsense Implications of Negation and Contradiction (2021.naacl-main)
Copied to clipboard
| Challenge: | a new commonsense knowledge graph for negated and contradicted events is developed to help humans reason about their underlying causes and effects. |
| Approach: | They propose a new commonsense knowledge graph with 624K if-then rules focusing on negated and contradictory events. |
| Outcome: | The proposed model can be used to analyze negated and contradicted statements in natural language. |
Counterfactual Story Reasoning and Generation (D19-1)
Copied to clipboard
| Challenge: | a desired property of AI systems is counterfactual reasoning: ability to predict causal changes in future events. |
| Approach: | They propose to rewrite a short story and a counterfactual event to make it compatible with the given counterfact. |
| Outcome: | The proposed task requires deep understanding of causal narrative chains and counterfactual invariance . the proposed dataset includes 81,407 counterfact "branches" without a rewritten storyline . |
DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts (2021.acl-long)
Copied to clipboard
Alisa Liu, Maarten Sap, Ximing Lu, Swabha Swayamdipta, Chandra Bhagavatula, Noah A. Smith, Yejin Choi
| Challenge: | Decoding-time Experts is a decoding- time method for controlled text generation . it combines a pretrained language model with "expert" LMs and/or "anti-expert" experts . |
| Approach: | They propose a decoding-time method that combines a pretrained language model with "expert" LMs and/or "anti-expert" experts to generate controlled text. |
| Outcome: | The proposed method outperforms existing controllable generation methods on automatic and human evaluations. |
CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoning (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies show that pre-trained language models perform well on commonsense-reasoning benchmark datasets, but building machines with commonsence to compose plausible sentences remains challenging. |
| Approach: | They propose a constrained text generation task for generative commonsense reasoning that generates a coherent sentence using common concepts. |
| Outcome: | The proposed task generates a coherent sentence describing an everyday scenario using common concepts over 35k concept-sets. |
“You Are An Expert Linguistic Annotator”: Limits of LLMs as Analyzers of Abstract Meaning Representation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) demonstrate proficiency and fluency in the use of language, but do they have the linguistic knowledge to serve as an expert linguistic annotator? |
| Approach: | They examine the successes and limitations of large language models using the Abstract Meaning Representation (AMR) parsing formalism. |
| Outcome: | The proposed models can reproduce the basic format of AMR, as well as some core event, argument, and modifier structure, but they have virtually no fully accurate parses. |
Reflective Decoding: Beyond Unidirectional Generation with Off-the-Shelf Language Models (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for generating text are unsupervised and require supervision. |
| Approach: | They propose an unsupervised method that uses two off-the-shelf pretrained LMs in opposite directions to apply them to non-sequential tasks. |
| Outcome: | The proposed method outperforms strong unsupervised baselines on paraphrasing and abductive text infilling. |
NovaCOMET: Open Commonsense Foundation Models with Symbolic Knowledge Distillation (2023.findings-emnlp)
Copied to clipboard
Peter West, Ronan Bras, Taylor Sorensen, Bill Lin, Liwei Jiang, Ximing Lu, Khyathi Chandu, Jack Hessel, Ashutosh Baheti, Chandra Bhagavatula, Yejin Choi
| Challenge: | a new commonsense knowledge model, NovaCOMET, combines knowledge and general task models. |
| Approach: | They propose an open commonsense knowledge model that combines knowledge and general task models. |
| Outcome: | The proposed model matches or exceeds existing knowledge models on commonsense reasoning tasks. |
Construction of the Literature Graph in Semantic Scholar (N18-3)
Copied to clipboard
Waleed Ammar, Dirk Groeneveld, Chandra Bhagavatula, Iz Beltagy, Miles Crawford, Doug Downey, Jason Dunkelberger, Ahmed Elgohary, Sergey Feldman, Vu Ha, Rodney Kinney, Sebastian Kohlmeier, Kyle Lo, Tyler Murray, Hsu-Han Ooi, Matthew Peters, Joanna Power, Sam Skjonsberg, Lucy Lu Wang, Chris Wilhelm, Zheng Yuan, Madeleine van Zuylen, Oren Etzioni
| Challenge: | Fig. 1 summarizes a scalable system for organizing published scientific literature into a heterogeneous graph . authors describe methods used to enable semantic features in www.semanticscholar.org . |
| Approach: | They describe a scalable system for organizing published scientific literature into a heterogeneous graph to facilitate algorithmic manipulation and discovery. |
| Outcome: | The proposed system can be deployed on a scalable platform and report empirical results for each task. |
Penguins Don’t Fly: Reasoning about Generics through Instantiations and Exceptions (2023.eacl-main)
Copied to clipboard
| Challenge: | Generics express generalizations about the world that are not universally true . commonsense knowledge bases encode some generic knowledge but rarely enumerate exceptions . |
| Approach: | They propose a framework informed by linguistic theory to generate exemplars for generics . they generate 19k exemplar cases for 650 generics and show they outperform a strong baseline . |
| Outcome: | The proposed framework outperforms a baseline framework by 12.8 precision points. |
I2D2: Inductive Knowledge Distillation with NeuroLogic and Self-Imitation (2023.acl-long)
Copied to clipboard
Chandra Bhagavatula, Jena D. Hwang, Doug Downey, Ronan Le Bras, Ximing Lu, Lianhui Qin, Keisuke Sakaguchi, Swabha Swayamdipta, Peter West, Yejin Choi
| Challenge: | Empirical results suggest that scale is not the only way to build commonsense capabilities. |
| Approach: | They propose a commonsense distillation framework that can achieve a competitive level of commonsensing without relying on the benefits of scale. |
| Outcome: | The proposed framework breaks the dependence on the extreme-scale teacher model with two innovations: (1) the novel adaptation of NeuroLogic Decoding to enhance the generation quality of the weak, off-the-shelf language models, and (2) self-imitation learning to iteratively learn from the model’s own enhanced commonsense acquisition capabilities. |
Natural Language Rationales with Full-Stack Visual Reasoning: From Pixels to Semantic Frames to Commonsense Graphs (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models that use natural language rationales provide intuitive, higher-level explanations that are easily understandable by humans. |
| Approach: | They propose a model that generates free-text rationales by combining pretrained language models with object recognition, grounded visual semantic frames, and visual commonsense graphs. |
| Outcome: | The proposed model generates free-text rationales by combining pretrained language models with object recognition, grounded visual semantic frames, and visual commonsense graphs. |
Content-Based Citation Recommendation (N18-1)
Copied to clipboard
| Challenge: | Existing citation recommendation systems rely on information of query documents such as author names and publication venue. |
| Approach: | They propose a content-based method for recommending citations in academic paper drafts . they embed a given query document into a vector space and use its nearest neighbors as candidates . |
| Outcome: | The proposed method outperforms published methods on PubMed and DBLP datasets without metadata. |
ACCoRD: A Multi-Document Approach to Generating Diverse Descriptions of Scientific Concepts (2022.emnlp-demos)
Copied to clipboard
Sonia Murthy, Kyle Lo, Daniel King, Chandra Bhagavatula, Bailey Kuehl, Sophie Johnson, Jonathan Borchardt, Daniel Weld, Tom Hope, Doug Downey
| Challenge: | Current systems that automatically define unfamiliar terms only surface a single "best" description for all users, which may not be accessible for all readers, given varying background knowledge. |
| Approach: | They propose an end-to-end system that generates sets of descriptions of scientific concepts . ACCoRD corpus includes 1,275 labeled contexts and 1,787 expert-authored concept descriptions . |
| Outcome: | The proposed system produces diverse descriptions of concepts in terms of reference concepts. |
GEMv2: Multilingual NLG Benchmarking in a Single Line of Code (2022.emnlp-demos)
Copied to clipboard
Sebastian Gehrmann, Abhik Bhattacharjee, Abinaya Mahendiran, Alex Wang, Alexandros Papangelis, Aman Madaan, Angelina Mcmillan-major, Anna Shvets, Ashish Upadhyay, Bernd Bohnet, Bingsheng Yao, Bryan Wilie, Chandra Bhagavatula, Chaobin You, Craig Thomson, Cristina Garbacea, Dakuo Wang, Daniel Deutsch, Deyi Xiong, Di Jin, Dimitra Gkatzia, Dragomir Radev, Elizabeth Clark, Esin Durmus, Faisal Ladhak, Filip Ginter, Genta Indra Winata, Hendrik Strobelt, Hiroaki Hayashi, Jekaterina Novikova, Jenna Kanerva, Jenny Chim, Jiawei Zhou, Jordan Clive, Joshua Maynez, João Sedoc, Juraj Juraska, Kaustubh Dhole, Khyathi Raghavi Chandu, Laura Perez Beltrachini, Leonardo F . R. Ribeiro, Lewis Tunstall, Li Zhang, Mahim Pushkarna, Mathias Creutz, Michael White, Mihir Sanjay Kale, Moussa Kamal Eddine, Nico Daheim, Nishant Subramani, Ondrej Dusek, Paul Pu Liang, Pawan Sasanka Ammanamanchi, Qi Zhu, Ratish Puduppully, Reno Kriz, Rifat Shahriyar, Ronald Cardenas, Saad Mahamood, Salomey Osei, Samuel Cahyawijaya, Sanja Štajner, Sebastien Montella, Shailza Jolly, Simon Mille, Tahmid Hasan, Tianhao Shen, Tosin Adewumi, Vikas Raunak, Vipul Raheja, Vitaly Nikolaev, Vivian Tsai, Yacine Jernite, Ying Xu, Yisi Sang, Yixin Liu, Yufang Hou
| Challenge: | Evaluations in machine learning rarely use the latest metrics, datasets, or human evaluation in favor of remaining compatible with prior work. |
| Approach: | They propose to use the Generation, Evaluation, and Metrics Benchmark to integrate new evaluation methods into existing evaluations. |
| Outcome: | The proposed evaluation infrastructure bridges the gap between the advantages of leaderboards and in-depth and evolving evaluations by allowing model developers to benefit from each other's work. |
Back to the Future: Unsupervised Backprop-based Decoding for Counterfactual and Abductive Commonsense Reasoning (2020.emnlp-main)
Copied to clipboard
Lianhui Qin, Vered Shwartz, Peter West, Chandra Bhagavatula, Jena D. Hwang, Ronan Le Bras, Antoine Bosselut, Yejin Choi
| Challenge: | Existing methods for integrating past and future contexts are limited and require manual input. |
| Approach: | They propose an unsupervised decoding algorithm that incorporates past and future contexts using off-the-shelf, left-to-right language models and no supervision. |
| Outcome: | The proposed method outperforms unsupervised methods on abductive and counterfactual reasoning tasks. |
On-the-Fly Attention Modulation for Neural Generation (2021.findings-acl)
Copied to clipboard
Yue Dong, Chandra Bhagavatula, Ximing Lu, Jena D. Hwang, Antoine Bosselut, Jackie Chi Kit Cheung, Yejin Choi
| Challenge: | Degeneration of neural text is associated with insufficient learning of task-specific characteristics by the attention mechanism. |
| Approach: | They propose to use attention modulation to inject priors into inference to improve fluency, creativity, and commonsense reasoning in neural text generation models. |
| Outcome: | The proposed method improves fluency, creativity, and commonsense reasoning, and significantly reduces sentence-level repetition. |
Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanations (2022.emnlp-main)
Copied to clipboard
Jaehun Jung, Lianhui Qin, Sean Welleck, Faeze Brahman, Chandra Bhagavatula, Ronan Le Bras, Yejin Choi
| Challenge: | Pre-trained language models struggle with consistent reasoning, and prompting methods are often noisy and inconsistent. |
| Approach: | They propose a few-shot inference method inspired by the Socratic way of conversation that generates a tree of explanations that bear logical relations between each other and frames it as a satisfiability problem. |
| Outcome: | The proposed method achieves 20% better accuracy than state-of-the-art prompting methods and performs competitively with supervised models. |
Information-Guided Identification of Training Data Imprint in (Proprietary) Large Language Models (2025.naacl-long)
Copied to clipboard
Abhilasha Ravichander, Jillian Fisher, Taylor Sorensen, Ximing Lu, Maria Antoniak, Bill Yuchen Lin, Niloofar Mireshghallah, Chandra Bhagavatula, Yejin Choi
| Challenge: | Lack of transparency in training data is limiting external oversight and inspection of LLMs for issues such as copyright infringement and data contamination. |
| Approach: | They propose a method to identify training data known to proprietary LLMs without requiring access to model weights or token probabilities by using information-guided probes. |
| Outcome: | The proposed method can identify training data known to proprietary LLMs without access to model weights or token probabilities. |
ClarifyDelphi: Reinforced Clarification Questions with Defeasibility Rewards for Social and Moral Situations (2023.acl-long)
Copied to clipboard
Valentina Pyatkin, Jena D. Hwang, Vivek Srikumar, Ximing Lu, Liwei Jiang, Yejin Choi, Chandra Bhagavatula
| Challenge: | Changing contexts can flip the moral judgment of an action. |
| Approach: | They propose an interactive system that learns to ask clarification questions to elicit salient contexts of a social or moral situation. |
| Outcome: | The proposed system generates more relevant, informative and defeasible questions compared to baselines. |
Symbolic Knowledge Distillation: from General Language Models to Commonsense Models (2022.naacl-main)
Copied to clipboard
Peter West, Chandra Bhagavatula, Jack Hessel, Jena Hwang, Liwei Jiang, Ronan Le Bras, Ximing Lu, Sean Welleck, Yejin Choi
| Challenge: | Prior studies suggested pre-trained language models possess limited understanding of commonsense knowledge despite otherwise stellar performance on leaderboards. |
| Approach: | They propose a framework that uses larger models to teach smaller models by distilling knowledge symbolically as text in addition to the neural model. |
| Outcome: | The proposed framework is based on a general language model teacher's commonsense knowledge graphs and a neural commonsensing model surpassing the teacher model's in all three criteria. |