Papers by Nathanael Chambers
CaT-Bench: Benchmarking Language Model Understanding of Causal and Temporal Dependencies in Plans (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on reasoning in plans focus on classical problems, simulated environments, or restricted language such as PDDL, but real-world plans cannot be tested to test for correctness and reliability. |
| Approach: | They propose a benchmark question that tests whether a step must necessarily occur before or after another in cooking recipe plans. |
| Outcome: | The proposed question-driven evaluation shows that SOTA LLMs are underwhelming and biased towards predicting dependence more often, but the best F1 result is 0.73. |
Toward Diverse Precondition Generation (2021.starsem-1)
Copied to clipboard
| Challenge: | a typical goal for language understanding is to logically connect the events of a discourse, but connective events are not described due to their commonsense nature. |
| Approach: | They propose a system that generates unique and diverse preconditions by using an event sampler, candidate generator, and post-processor. |
| Outcome: | The proposed system can generate unique and diverse preconditions without training on diverse examples. |
Using Commonsense Knowledge to Answer Why-Questions (2022.emnlp-main)
Copied to clipboard
Yash Kumar Lal, Niket Tandon, Tanvi Aggarwal, Horace Liu, Nathanael Chambers, Raymond Mooney, Niranjan Balasubramanian
| Challenge: | Existing approaches to integrating commonsense knowledge into large language models are implicit and explicit. |
| Approach: | They analyze the effects of model size and methods of injecting knowledge into TellMeWhy datasets to determine what aspects of commonsense knowledge are available in large language models. |
| Outcome: | The largest models yield substantial improvements over base models, but the amount of improvement decreases with larger model size. |
Hierarchical Quantized Representations for Script Generation (D18-1)
Copied to clipboard
| Challenge: | Scripts define knowledge about how everyday scenarios are expected to unfold . language models tend towards local coherency, which is a major issue . |
| Approach: | They propose an autoencoder model with a latent space defined by a hierarchy of categorical variables . they use a vector quantization based approach which allows continuous embeddings to be associated with each latent variable value . |
| Outcome: | The proposed model outperforms a language modeling-based method on several tasks and lowers perplexity scores. |
Modeling Label Semantics for Predicting Emotional Reactions (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for predicting how events induce emotions ignore the semantics of the labels themselves. |
| Approach: | They propose that the semantics of emotion labels can guide a model’s attention when representing the input story. |
| Outcome: | The proposed model can model the semantics of emotion labels and track correlations on unlabeled data. |
Modeling Preconditions in Text with a Crowd-sourced Dataset (2020.findings-emnlp)
Copied to clipboard
Heeyoung Kwon, Mahnaz Koupaee, Pratyush Singh, Gargi Sawhney, Anmol Shukla, Keerthi Kumar Kallur, Nathanael Chambers, Niranjan Balasubramanian
| Challenge: | Existing methods for modeling preconditions in text are limited due to the lack of large scale labeled data grounded in text. |
| Approach: | They propose a crowd-sourced annotation of preconditions between event pairs in newswire that is larger than prior annotations. |
| Outcome: | The proposed model outperforms existing models on two task sets, showing that precondition knowledge is not easily accessible in LM-derived representations alone. |
PASTA: A Dataset for Modeling PArticipant STAtes in Narratives (2023.tacl-1)
Copied to clipboard
Sayontan Ghosh, Mahnaz Koupaee, Isabella Chen, Francis Ferraro, Nathanael Chambers, Niranjan Balasubramanian
| Challenge: | Existing models that understand narratives should infer these implicit states and their causal relationships with the narrative's explicit events. |
| Approach: | They propose a dataset that contains inferable participant states, a counterfactual perturbation to each state and the changes to the story that would be necessary if the counterfact was true. |
| Outcome: | The proposed model can reason about the impact of changes to the story that would be necessary if the counterfactual were true. |
Detecting Denial-of-Service Attacks from Social Media Text: Applying NLP to Computer Security (N18-1)
Copied to clipboard
| Challenge: | Distributed Denial of Service (DDoS) attacks are becoming more frequent and more severe in their impact. |
| Approach: | They propose a feed-forward neural network and a partially labeled LDA model that use social media as an indirect measure of network service status. |
| Outcome: | The proposed model outperforms previous work by significant margins and provides the first fine-grained analysis of how the public reacts to ongoing network attacks. |
TellMeWhy: A Dataset for Answering Why-Questions in Narratives (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing models do not have the ability to answer "why" questions that require commonsense knowledge external to the narrative. |
| Approach: | They propose a crowd-sourced dataset that asks why characters perform actions . they show that state-of-the-art models are far below human performance on answering such questions . |
| Outcome: | The proposed dataset shows that state-of-the-art models are far below human performance on answering such questions. |
Character-Based Models for Adversarial Phone Extraction: Preventing Human Sex Trafficking (D19-55)
Copied to clipboard
Nathanael Chambers, Timothy Forman, Catherine Griswold, Kevin Lu, Yogaish Khastgir, Stephen Steckler
| Challenge: | Illicit activity on the Web often obscures information between client and seller, such as the seller’s phone number. |
| Approach: | They propose to use a dataset to model adversarial noise in a text extraction system and propose a visual character language model to interpret unseen unicode characters. |
| Outcome: | The proposed model improves number recognition by 89% over a CRF with a CNN and shows that unicode characters can be translated to unicoding. |
Connecting the Dots: Event Graph Schema Induction with Path Language Modeling (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to automate event extraction focus on uncertainty, re-occurring events and multiple hypotheses. |
| Approach: | They propose a new Event Graph Schema where two event types are connected through multiple paths involving entities that fill important roles in a coherent story. |
| Outcome: | The proposed model is highly effective at inducing salient and coherent schemas. |
Conditional Generation of Temporally-ordered Event Sequences (2021.acl-long)
Copied to clipboard
| Challenge: | a new model of narrative schema knowledge does not capture the temporal relationships between events . a temporal order model is able to unscramble event sequences without access to labeled temporal training data . |
| Approach: | They propose a temporal order-based model that can be flexibly applied to different tasks . they use a BART-based conditional generation model that captures temporal co-occurrence . |
| Outcome: | The proposed model outperforms existing models on temporal ordering and event infilling tasks. |
Don’t Let Discourse Confine Your Model: Sequence Perturbations for Improved Event Language Models (2021.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to train event language models on text constrain them to follow discourse order of events. |
| Approach: | They propose a method to perturb event sequences so that they can relax model dependence on text order. |
| Outcome: | The proposed technique improves performance on applications and out-of-domain events data. |
SAGEViz: SchemA GEneration and Visualization (2023.emnlp-demo)
Copied to clipboard
Sugam Devare, Mahnaz Koupaee, Gautham Gunapati, Sayontan Ghosh, Sai Vallurupalli, Yash Kumar Lal, Francis Ferraro, Nathanael Chambers, Greg Durrett, Raymond Mooney, Katrin Erk, Niranjan Balasubramanian
| Challenge: | Schema induction involves creating a graph representation depicting how events unfold . supervised and few-shot approaches are not scalable and time-consuming . |
| Approach: | They propose a tool that utilizes human-AI collaboration to create and update complex schema graphs efficiently. |
| Outcome: | The proposed tool can generate schemas of better quality and be used by users in a variety of domains. |
Modeling Complex Event Scenarios via Simple Entity-focused Questions (2023.eacl-main)
Copied to clipboard
| Challenge: | Event schemas describe a sequence of events in a particular context, but they are difficult to model with standard event language models. |
| Approach: | They propose a question-guided generation framework that generates events as answers to questions about participants. |
| Outcome: | The proposed framework provides better coverage of participants, diverse events within a domain, comparable perplexities for modeling event sequences, and more effective control for interactive schema generation. |
Causal Graph based Event Reasoning using Semantic Relation Experts (2025.acl-long)
Copied to clipboard
| Challenge: | Recent advances in event reasoning have limited ability to accurately identify causal connections between events. |
| Approach: | They propose a collaborative approach to generate correct graphs and graphs to assist reasoning . they propose 'a causal chain of events' task that requires a causal link between events . |
| Outcome: | The proposed approach achieves competitive results with state-of-the-art models on forecasting and next event prediction tasks. |