| Challenge: | Existing time series captioning benchmarks rely on fully synthetic or generic captions . authors propose a pipeline for generating high-fidelity synthetic captions, which is validated . |
| Approach: | They propose a benchmark for Context-aware Time Series reasoning across 11 diverse domains . they evaluate leading Vision-Language Models on their benchmark . |
| Outcome: | The proposed benchmark evaluates 1746 human-rewritten captions and shows they perform better than open-source models. |
Similar Papers
Language Models Still Struggle to Zero-shot Reason about Time Series (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Time series are critical for decision-making in fields like finance and healthcare. |
| Approach: | They propose a framework for time series reasoning that includes formal tasks and a dataset of multi-scale time series paired with text captions across ten domains. |
| Outcome: | The proposed framework combines formal tasks and a dataset of multi-scale time series paired with text captions across ten domains to examine whether language models achieve three forms of reasoning. |
Evaluating Large Language Models on Time Series Feature Understanding: A Comprehensive Taxonomy and Benchmark (2024.emnlp-main)
Copied to clipboard
Elizabeth Fons, Rachneet Kaur, Soham Palande, Zhen Zeng, Tucker Balch, Manuela Veloso, Svitlana Vyetrenko
| Challenge: | Large Language Models (LLMs) are a critical tool for time series analysis and reporting in many fields, including healthcare, finance, climate, and many more. |
| Approach: | They propose a framework for rigorously evaluating the capabilities of Large Language Models (LLMs) on time series understanding, encompassing both univariate and multivariate forms. |
| Outcome: | The proposed framework delineates various characteristics inherent in time series data. |
LLaTiSA: Towards Difficulty-Stratified Time Series Reasoning from Visual Perception to Semantics (2026.findings-acl)
Copied to clipboard
| Challenge: | Current research hinders the development of unified Time Series Reasoning Models (TSRMs) time series data are a fundamental modality for capturing the temporal dynamics of complex systems. |
| Approach: | They propose a time series reasoning model that integrates visualized patterns with precision-calibrated numerical tables to enhance the temporal perception of Vision-Language Models. |
| Outcome: | The proposed model outperforms existing models and exhibits robust out-of-distribution generalization across diverse tasks and real-world scenarios. |
Can Large Language Models Adequately Perform Symbolic Reasoning Over Time Series? (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) and Multimodal LLMs (MLLMs) show strong performance in complex reasoning tasks, but their ability to extract symbolic laws from time series data remains underexplored. |
| Approach: | They propose a benchmark to assess symbolic reasoning over real-world time series across three tasks: multivariate symbolic regression, Boolean network inference, and causal discovery. |
| Outcome: | The proposed framework integrates LLMs with genetic programming to form a closed-loop symbolic reasoning system. |
TimeBench: A Comprehensive Evaluation of Temporal Reasoning Abilities in Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Grasping the concept of time is a fundamental facet of human cognition. |
| Approach: | They propose a hierarchical temporal reasoning benchmark that covers a broad spectrum of temporal phenomena. |
| Outcome: | The proposed benchmark shows that state-of-the-art LLMs are still far behind humans in temporal reasoning . |
Do Language Models Have a Common Sense regarding Time? Revisiting Temporal Commonsense Reasoning in the Era of Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Temporal reasoning is a vital component of human communication and understanding, yet remains an underexplored area within the context of Large Language Models (LLMs). |
| Approach: | They propose to use 3 prompting strategies to evaluate 8 different LLMs across 6 datasets and 2 Code Generation LMs to perform the analysis. |
| Outcome: | The proposed models perform better on NLP tasks than the standard models on the same dataset. |
Truth-Conditional Captions for Time Series Data (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing models with attention mechanisms can generate fluent descriptions of salient patterns in time series, but they often generate factually incorrect descriptions. |
| Approach: | They propose a model which first runs small learned programs on the input time series, then identifies the programs/patterns which hold true for the given input, and finally conditions on *only* the chosen valid program to generate the output text description. |
| Outcome: | The proposed model extracts high-level patterns from the data and generates high precision captions even though it is built on a small space of modules. |
ChronoSense: Exploring Temporal Understanding in Large Language Models with Time Intervals of Events (2025.acl-short)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) still face significant challenges in reasoning and arithmetic. |
| Approach: | They propose a new benchmark to evaluate LLMs' temporal understanding that includes 16 tasks identifying the Allen relation between two temporal events and temporal arithmetic. |
| Outcome: | The proposed model handles Allen relations, even symmetrical ones, quite differently. |
CaT-Bench: Benchmarking Language Model Understanding of Causal and Temporal Dependencies in Plans (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on reasoning in plans focus on classical problems, simulated environments, or restricted language such as PDDL, but real-world plans cannot be tested to test for correctness and reliability. |
| Approach: | They propose a benchmark question that tests whether a step must necessarily occur before or after another in cooking recipe plans. |
| Outcome: | The proposed question-driven evaluation shows that SOTA LLMs are underwhelming and biased towards predicting dependence more often, but the best F1 result is 0.73. |
ChiKhaPo: A Large-Scale Multilingual Benchmark for Evaluating Lexical Comprehension and Generation in Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for large language models (LLMs) are restricted to high- or mid-resource languages, and evaluate performance on higher-order tasks in reasoning and generation. |
| Approach: | They propose a multilingual benchmarking tool to evaluate lexical comprehension and generation abilities of large language models. |
| Outcome: | The proposed benchmarks cover 2700+ languages and surpasses existing benchmarks in terms of language coverage. |