Papers by Qiang Ning
Event Time Extraction and Propagation via Graph Attention Networks (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing work on grounding events into a precise timeline has been limited due to the inherent ambiguity of language and the requirement for information propagation over inter-related events. |
| Approach: | They propose a 4-tuple temporal representation for entity slot filling to ground events into a timeline using a graph attention network approach. |
| Outcome: | The proposed approach yields 7.0% match rate over contextualized embedding approaches and 16.3% higher match rate compared to sentence-level manual event time argument annotation. |
“Going on a vacation” takes longer than “Going for a walk”: A Study of Temporal Commonsense Understanding (D19-1)
Copied to clipboard
| Challenge: | a new study examines temporal commonsense and compares it to human performance on a dataset . a previous study focused on duration, frequency, stationarity and ordering, but not all aspects of temporal similarity have been studied. |
| Approach: | They define five classes of temporal commonsense and use crowdsourcing to develop a new dataset that serves as a test set. |
| Outcome: | The proposed dataset shows that the best current methods are far behind human performance by 20%. |
Spatial and Temporal Language Understanding: Representation, Reasoning, and Grounding (2024.naacl-tutorials)
Copied to clipboard
| Challenge: | This tutorial provides an overview of cutting edge research on spatial and temporal language understanding. |
| Approach: | This tutorial provides an overview of cutting edge research on spatial and temporal language understanding. |
| Outcome: | This tutorial provides an overview of cutting edge research on spatial and temporal language understanding. |
Temporal Common Sense Acquisition with Minimal Supervision (2020.acl-main)
Copied to clipboard
| Challenge: | Temporal common sense is crucial for understanding natural language, but its acquisition is challenging . human annotation on such concepts is costly and often not made explicit in text . |
| Approach: | They propose a method that exploits explicit and implicit mentions of temporal common sense to build a temporal similarity language model. |
| Outcome: | The proposed model gives better predictions of various dimensions of temporal common sense than the standard BERT. |
Event-Centric Natural Language Processing (2021.acl-tutorials)
Copied to clipboard
| Challenge: | This tutorial will provide an introduction to various methods for automating the extraction, conceptualization and prediction of events and their relations. |
| Approach: | This tutorial will provide an introduction to various methods for automating events and their relations, and a wide range of NLU and commonsense understanding tasks. |
| Outcome: | This tutorial will provide an introduction to various methods for automating extraction, conceptualization and prediction of events and their relations, and a wide range of NLU and commonsense understanding tasks. |
Temporal Reasoning on Implicit Events from Distant Supervision (2021.naacl-main)
Copied to clipboard
| Challenge: | a novel temporal reasoning dataset evaluates the degree to which systems understand implicit events . state-of-the-art models struggle when predicting temporal relationships between implicit and explicit events - a recent paper . |
| Approach: | They propose a temporal reasoning dataset that evaluates the degree to which systems understand implicit events. |
| Outcome: | The proposed model outperforms baseline systems on TRACIE by 5% and 11% on MATRES, an explicit event benchmark. |
Improving Temporal Relation Extraction with a Globally Acquired Statistical Resource (N18-1)
Copied to clipboard
| Challenge: | Existing temporal extraction systems that extract temporal relations can be improved by using a resource that provides prior knowledge of the temporal order that events usually follow. |
| Approach: | They propose to use a probabilistic knowledge base acquired in the news domain to extract temporal relations between events from the New York Times articles over a 20-year span. |
| Outcome: | The proposed system and resource are both publicly available. |
Extracting Temporal Event Relation with Syntax-guided Graph Transformer (2022.findings-naacl)
Copied to clipboard
| Challenge: | Temporal relationship extraction is crucial for understanding complex events and reasoning over them. |
| Approach: | They propose a Syntax-guided Graph Transformer network to extract temporal relations between events by explicitly exploiting the connection between two events based on their dependency parsing trees. |
| Outcome: | The proposed approach outperforms state-of-the-art methods on MATRES and TB-DENSE with up to 7.9% absolute F-score gain. |
Open Domain Question Answering with Conflicting Contexts (2025.findings-naacl)
Copied to clipboard
Siyi Liu, Qiang Ning, Kishaloy Halder, Zheng Qi, Wei Xiao, Phu Mon Htut, Yi Zhang, Neha Anna John, Bonan Min, Yassine Benajiba, Dan Roth
| Challenge: | Open domain question answering systems often rely on information retrieved from large collections of text to answer questions. |
| Approach: | They evaluate and benchmark three powerful Large Language Models with a dataset . they find that 25% of unambiguous open domain questions can lead to conflicting contexts . |
| Outcome: | The proposed model can't be used to answer questions with conflicting contexts . it can be fine tuned to provide richer information into the model's training . |
Joint Event and Temporal Relation Extraction with Shared Representations and Structured Prediction (D19-1)
Copied to clipboard
| Challenge: | Existing systems treat this task as a pipeline of two separate subtasks, i.e., event extraction and temporal relation classification. |
| Approach: | They propose a joint event and temporal relation extraction model with shared representation learning and structured prediction. |
| Outcome: | The proposed method improves both event extraction and temporal relation extraction over state-of-the-art systems. |
PInKS: Preconditioned Commonsense Inference with Minimal Supervision (2022.aacl-main)
Copied to clipboard
| Challenge: | Existing models for reasoning with preconditions lack data on the problem and lack of support for such reasoning. |
| Approach: | They propose to improve the model for reasoning with preconditions through minimum supervision by PAC-Bayesian informativeness analysis and precision measures. |
| Outcome: | The proposed model improves on benchmarks focused on reasoning with the preconditions of commonsense knowledge (up to 40% Macro-F1 scores) it also improves inferences on PAC-Bayesian informativeness analysis, precision measures, and ablation studies. |
QuASE: Question-Answer Driven Sentence Encoding (2020.acl-main)
Copied to clipboard
| Challenge: | Question-answering (QA) data often encodes essential information in many facets . a growing interest of QA has led to many large-scale QA datasets available to the community . |
| Approach: | They propose a question-answer driven sentence encoding framework to learn representations from QA data. |
| Outcome: | The proposed framework learns representations from QA data, using BERT or other state-of-the-art contextual language models. |
CogCompTime: A Tool for Understanding Time in Natural Language (D18-2)
Copied to clipboard
| Challenge: | Existing systems that extract temporal information from text can be useful for natural language understanding. |
| Approach: | They propose a system that extracts temporal information from text and normalizes it to a standard format. |
| Outcome: | The proposed system achieves state-of-the-art performance and incorporates the most recent progress. |
Easy, Reproducible and Quality-Controlled Data Collection with CROWDAQ (2020.emnlp-demos)
Copied to clipboard
Qiang Ning, Hao Wu, Pradeep Dasigi, Dheeru Dua, Matt Gardner, Robert L. Logan IV, Ana Marasović, Zhen Nie
| Challenge: | Efficient data collection is important for advancing research and building time-sensitive applications. |
| Approach: | They propose an open-source platform that standardizes the data collection pipeline . it includes customizable user interface components, automated annotator qualification, and saved pipelines . |
| Outcome: | The proposed platform simplifies data annotation significantly on diverse datasets . it can be used by researchers and engineers to improve reproducibility and minimize overhead . |
Partial Or Complete, That’s The Question (N19-1)
Copied to clipboard
| Challenge: | Existing annotation schemes aim at acquiring completely annotated structures, but partial annotations can be costly and hinder learning. |
| Approach: | They propose a method to find out that learning from partial structures can sometimes outperform learning from complete ones. |
| Outcome: | The proposed method outperforms existing methods in three different structured learning tasks. |
Aligning to Constraints for Data-Efficient Language Model Customization (2025.findings-naacl)
Copied to clipboard
Fei Wang, Chao Shang, Shuai Wang, Sarthak Jain, Qiang Ning, Bonan Min, Vittorio Castelli, Yassine Benajiba, Dan Roth
| Challenge: | General-purpose language models (LMs) are aligned to diverse user intents, but fall short when it comes to specific applications. |
| Approach: | They propose a framework that uses constraints to automatically produce supervision signals for user alignment with constraints. |
| Outcome: | The proposed framework can produce supervision signals for user alignment with constraints. |
ESTER: A Machine Reading Comprehension Dataset for Reasoning about Event Semantic Relations (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent event-centric reading comprehension datasets focus mostly on event arguments or temporal relations. |
| Approach: | They propose a machine reading comprehension dataset that leverages natural language queries to reason about the five most common event semantic relations. |
| Outcome: | The proposed dataset shows that current SOTA systems achieve 22.1%, 63.3% and 83.5% for token-based exact-match, **F1** and event-based **HIT@1** scores. |
SPARTQA: A Textual Question Answering Benchmark for Spatial Reasoning (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing studies have focused on the spatial reasoning capabilities of modern language models (LMs) however, there has been limited research into the spatial thinking capabilities of LMs. |
| Approach: | They propose a question-answering (QA) benchmark for spatial reasoning on natural language text which contains more realistic spatial phenomena not covered by prior work. |
| Outcome: | The proposed method significantly improves LMs' ability on spatial understanding, which in turn helps solve two external datasets, bAbI, and boolQ. |
Evaluating Models’ Local Decision Boundaries via Contrast Sets (2020.findings-emnlp)
Copied to clipboard
Matt Gardner, Yoav Artzi, Victoria Basmov, Jonathan Berant, Ben Bogin, Sihao Chen, Pradeep Dasigi, Dheeru Dua, Yanai Elazar, Ananth Gottumukkala, Nitish Gupta, Hannaneh Hajishirzi, Gabriel Ilharco, Daniel Khashabi, Kevin Lin, Jiangming Liu, Nelson F. Liu, Phoebe Mulcaire, Qiang Ning, Sameer Singh, Noah A. Smith, Sanjay Subramanian, Reut Tsarfaty, Eric Wallace, Ally Zhang, Ben Zhou
| Challenge: | Standard test sets for supervised learning evaluate in-distribution generalization but are misleading when a dataset has systematic gaps. |
| Approach: | They propose a more rigorous annotation paradigm for NLP that helps to close systematic gaps in the test data. |
| Outcome: | The proposed model performs significantly lower on contrast sets than on the original test sets—up to 25% in some cases. |
Answer Consolidation: Formulation and Benchmarking (2022.naacl-main)
Copied to clipboard
| Challenge: | Current question answering systems assume each question to have one correct answer. |
| Approach: | They propose a problem where answers are partitioned into multiple groups . they construct a comprehensive and non-redundant set of answers by picking one answer from each group . |
| Outcome: | The proposed model performs better than previous models, but it needs further improvements. |
TORQUE: A Reading Comprehension Dataset of Temporal Ordering Questions (2020.emnlp-main)
Copied to clipboard
| Challenge: | Current machine reading comprehension benchmarks have no questions that test temporal phenomena . a new study studies reading comprehension for temporal relations . |
| Approach: | They propose a reading comprehension benchmark built on news snippets and 21k human-generated questions querying temporal relationships. |
| Outcome: | The new reading comprehension benchmark TORQUE achieves an exact-match score of 51% on the test set . the benchmark is built on 3.2k news snippets with 21k human-generated questions . |
A Meta-framework for Spatiotemporal Quantity Extraction from Text (2022.acl-long)
Copied to clipboard
| Challenge: | a meta-framework for news events that extracts quantities from text is proposed . a previous work on news events focused on extracting event mentions, attributes, and relationships . |
| Approach: | They propose a meta-framework for solving the NLP problem of spatiotemporal quantity extraction . they demonstrate the framework is general and extensible, and shareable crowdsourcing pipeline and baseline models are used . |
| Outcome: | The proposed framework is general and extensible, the authors say . it can extract quantity from news streams, quickly respond to emergencies, investigate incidents . |
A Multi-Axis Annotation Scheme for Event Temporal Relations (P18-1)
Copied to clipboard
| Challenge: | Existing temporal relation (TempRel) annotation schemes have low inter-annotator agreements even between experts, suggesting that the current annotation task needs a better definition. |
| Approach: | They propose to annotate temporal relation (TempRel) annotation schemes based on event start-points instead of a conventional 60’s-80’s model. |
| Outcome: | The proposed model improves IAA from the conventional 60’s to 80’s and can be used by crowdsourcing to alleviate labor intensity. |
Foreseeing the Benefits of Incidental Supervision (2021.emnlp-main)
Copied to clipboard
| Challenge: | Real-world applications often require improved models by leveraging a range of cheap incidental supervision signals. |
| Approach: | They propose a unified PAC-Bayesian motivated informativeness measure that characterizes the uncertainty reduction provided by incidental supervision signals. |
| Outcome: | The proposed measure quantifies the value added by incidental supervision signals to sequence tagging tasks. |
An Improved Neural Baseline for Temporal Relation Extraction (D19-1)
Copied to clipboard
| Challenge: | Existing datasets are small and/or have low inter-annotator agreements. |
| Approach: | They propose a new neural system that achieves 10% absolute accuracy improvement over the previous best system. |
| Outcome: | The proposed system achieves 10% absolute improvement over the previous best system on two benchmark datasets. |
Joint Reasoning for Temporal and Causal Relations (P18-1)
Copied to clipboard
| Challenge: | a cause must occur earlier than its effect, temporal and causal relations are closely related . a joint inference framework is developed for studying temporal, causal relations . |
| Approach: | They propose a joint inference framework for temporal and causal relations . they use constraints inherent in time and causality to enforce constraints . |
| Outcome: | The proposed framework improves extraction of temporal and causal relations from text. |
CogCompNLP: Your Swiss Army Knife for NLP (L18-1)
Copied to clipboard
Daniel Khashabi, Mark Sammons, Ben Zhou, Tom Redman, Christos Christodoulopoulos, Vivek Srikumar, Nicholas Rizzolo, Lev Ratinov, Guanheng Luo, Quang Do, Chen-Tse Tsai, Subhro Roy, Stephen Mayhew, Zhili Feng, John Wieting, Xiaodong Yu, Yangqiu Song, Shashank Gupta, Shyam Upadhyay, Naveen Arivazhagan, Qiang Ning, Shaoshi Ling, Dan Roth
| Challenge: | a corpus-reader module supports popular corpora, feature extraction and annotation modules for semantic and syntactic tasks. |
| Approach: | They propose a library that provides modules to address different challenges . they provide a corpus-reader module that supports popular corpora in the NLP community . |
| Outcome: | The proposed library simplifies the process of design and development of NLP applications by providing modules to address different challenges. |
Indirectly Supervised Natural Language Processing (2023.acl-tutorials)
Copied to clipboard
| Challenge: | a tutorial on indirect supervision addresses challenges in ML for NLP . conventional approaches to NLP use taskspecific labeled examples of a large volume . indirect supervision is useful for a wide range of NLP tasks, but it is not enough for decoders . |
| Approach: | This tutorial aims to address questions about indirect supervision in machine learning . authors discuss indirect supervision from T′ that handles T with outputs spanning from a moderate size to an open space . |
| Outcome: | This tutorial aims to answer questions about how to provide supervision for ML tasks . it will discuss indirect supervision from T′ that handles T with outputs spanning from a moderate size to an open space . |