Toward Building a Language Model for Understanding Temporal Commonsense (2022.aacl-srw)
Copied to clipboard
| Challenge: | Pre-trained language models such as BERT are still poor in temporal reasoning . commonsense reasoning is crucial for natural language processing (NLP) |
| Approach: | They propose to use multi-step fine-tuning and masked language modeling to predict mangled temporal indicators that are crucial for commonsense reasoning. |
| Outcome: | The proposed model improves performance on multiple time-related tasks. |
Similar Papers
Do Language Models Have a Common Sense regarding Time? Revisiting Temporal Commonsense Reasoning in the Era of Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Temporal reasoning is a vital component of human communication and understanding, yet remains an underexplored area within the context of Large Language Models (LLMs). |
| Approach: | They propose to use 3 prompting strategies to evaluate 8 different LLMs across 6 datasets and 2 Code Generation LMs to perform the analysis. |
| Outcome: | The proposed models perform better on NLP tasks than the standard models on the same dataset. |
Neural Language Modeling for Contextualized Temporal Graph Generation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for temporal reasoning have been used for a number of applications, but their potential for tempor reasoning over event graphs has not been explored. |
| Approach: | They propose to use large-scale pre-trained language models to generate an event-level temporal graph from a document using existing IE/NLP tools. |
| Outcome: | The proposed method outperforms the closest existing method on several metrics on a hand-labeled, out-of-domain corpus. |
Multilingual Normalization of Temporal Expressions with Masked Language Models (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for normalizing temporal expressions are rule-based, which severely limits the applicability in multilingual settings. |
| Approach: | They propose a neural method for normalizing temporal expressions based on masked language modeling and a slot-based prediction scheme for context-independent representations. |
| Outcome: | The proposed method outperforms existing rule-based methods in many languages and in particular, for low-resource languages with performance improvements of up to 33 F1 on average compared to the state of the art. |
Time-Aware Language Models as Temporal Knowledge Bases (2022.tacl-1)
Copied to clipboard
Bhuwan Dhingra, Jeremy R. Cole, Julian Martin Eisenschlos, Daniel Gillick, Jacob Eisenstein, William W. Cohen
| Challenge: | Existing language models are trained on snapshots of data collected at a specific moment in time. |
| Approach: | They propose a diagnostic dataset aimed at probing LMs for factual knowledge that changes over time. |
| Outcome: | The proposed method improves memorization of seen facts and calibration on unseen facts from future time periods. |
Are Large Language Model Temporally Grounded? (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent large language models lack a consistent temporal model of textual narratives . sentence ordering in unlabelled texts is only weakly correlated with event ordering . |
| Approach: | They evaluate LLMs with textual narratives and evaluate their common-sense knowledge . they find that LLM models struggle the most with self-consistency . |
| Outcome: | The proposed models lack a consistent temporal model of textual narratives. |
Exploring Contextualized Neural Language Models for Temporal Dependency Parsing (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that deep contextualized language models (LMs) can extract temporal relations between events and time expressions. |
| Approach: | They propose a temporal relation extraction technique which extracts temporal relations between events and time expressions. |
| Outcome: | The proposed method significantly improves temporal dependency parsing, the authors show . their work compares the proposed method to other methods and shows where they may fail . |
CoCoLM: Complex Commonsense Enhanced Language Model with Discourse Relations (2022.findings-acl)
Copied to clipboard
| Challenge: | Large-scale pre-trained language models have demonstrated strong knowledge representation ability, but struggle with complex commonsense knowledge that involves multiple eventualities. |
| Approach: | They propose to help pre-trained language models better incorporate complex commonsense knowledge that involves multiple eventualities. |
| Outcome: | The proposed model can learn to use the memorized knowledge for different tasks and achieve outstanding performance on many downstream natural language processing (NLP) tasks. |
TIMEDIAL: Temporal Commonsense Reasoning in Dialog (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies on pre-trained language models for dialog reasoning fail to understand context correctly. |
| Approach: | They propose to use a crowd-sourced English task and a time-based task to test models' temporal reasoning abilities in dialogs. |
| Outcome: | The proposed task and crowd-sourced English challenge set show that even the best performing models struggle on this task compared to humans. |
ChronoSense: Exploring Temporal Understanding in Large Language Models with Time Intervals of Events (2025.acl-short)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) still face significant challenges in reasoning and arithmetic. |
| Approach: | They propose a new benchmark to evaluate LLMs' temporal understanding that includes 16 tasks identifying the Allen relation between two temporal events and temporal arithmetic. |
| Outcome: | The proposed model handles Allen relations, even symmetrical ones, quite differently. |
Inferring Events from Time Series using Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Prior work on reasoning about time series in conjunction with natural language has largely overlooked event descriptions and focused on tasks involving just numeric data like trend analysis or anomaly detection. |
| Approach: | They propose a method for generating tasks that test a model’s ability to reason about events associated with time series data based on sports data and develop a benchmarking method. |
| Outcome: | The proposed method can infer unobserved events from time series data, even when providing minimal context. |