Evaluating Large Language Models on Time Series Feature Understanding: A Comprehensive Taxonomy and Benchmark (2024.emnlp-main)
Copied to clipboard
Elizabeth Fons, Rachneet Kaur, Soham Palande, Zhen Zeng, Tucker Balch, Manuela Veloso, Svitlana Vyetrenko
| Challenge: | Large Language Models (LLMs) are a critical tool for time series analysis and reporting in many fields, including healthcare, finance, climate, and many more. |
| Approach: | They propose a framework for rigorously evaluating the capabilities of Large Language Models (LLMs) on time series understanding, encompassing both univariate and multivariate forms. |
| Outcome: | The proposed framework delineates various characteristics inherent in time series data. |
Similar Papers
On Evaluating LLMs’ Capabilities as Functional Approximators: A Bayesian Evaluation Framework (2025.coling-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have revolutionized the way we can formulate tasks in text-in-text-out format. |
| Approach: | They propose a new evaluation framework to comprehensively assess LLMs’ function modeling abilities by adopting a Bayesian perspective of function modeling. |
| Outcome: | The proposed evaluation framework enables LLMs to excel in utilizing prior knowledge to develop a strong understanding of the underlying function. |
Inferring Events from Time Series using Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Prior work on reasoning about time series in conjunction with natural language has largely overlooked event descriptions and focused on tasks involving just numeric data like trend analysis or anomaly detection. |
| Approach: | They propose a method for generating tasks that test a model’s ability to reason about events associated with time series data based on sports data and develop a benchmarking method. |
| Outcome: | The proposed method can infer unobserved events from time series data, even when providing minimal context. |
Analyzing Temporal Complex Events with Large Language Models? A Benchmark towards Temporal, Long Context Understanding (2024.acl-long)
Copied to clipboard
| Challenge: | Existing research in complex event analysis has made significant strides but is constrained by inadequate natural language processing techniques. |
| Approach: | They propose a novel approach using Large Language Models to extract and analyze the event chain within TCE, characterized by their key points and timestamps. |
| Outcome: | The proposed model performs comparable to models with long context window and retrieval-augmented generation method in three distinct tasks . |
Time-LlaMA: Adapting Large Language Models for Time Series Modeling via Dynamic Low-rank Adaptation (2025.acl-srw)
Copied to clipboard
| Challenge: | Recent studies have demonstrated that large language models possess robust pattern recognition and semantic understanding capabilities over time series data. |
| Approach: | They propose a time series model that converts time series input into token embeddings and aligns time sequence embeddables with text prompts. |
| Outcome: | The proposed framework achieves the state-of-the-art (SOTA) performance and has potentials for wide industrial usages. |
Remember This Event That Year? Assessing Temporal Information and Understanding in Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly ubiquitous, yet their ability to effectively retain and reason about temporal information remains limited. |
| Approach: | They propose six metrics to assess three learning paradigms to enhance temporal knowledge acquisition. |
| Outcome: | The proposed methods improve performance and reduce incorrect outputs. |
A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendations (2024.emnlp-main)
Copied to clipboard
Md Tahmid Rahman Laskar, Sawsan Alqahtani, M Saiful Bari, Mizanur Rahman, Mohammad Abdullah Matin Khan, Haidar Khan, Israt Jahan, Amran Bhuiyan, Chee Wei Tan, Md Rizwan Parvez, Enamul Hoque, Shafiq Joty, Jimmy Huang
| Challenge: | Large Language Models (LLMs) have gained significant attention due to their capabilities in performing diverse tasks across domains. |
| Approach: | They review the primary challenges and limitations causing inconsistencies in evaluations . early models could generate coherent text but limited to simple tasks . |
| Outcome: | The proposed evaluations are reproducible, reliable, and robust. |
Is Your LLM Outdated? A Deep Look at Temporal Generalization (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods to evaluate large language models are limited due to their inherent dynamic nature and the inherent dynamicity of language and information. |
| Approach: | They introduce a new evaluation framework that employs fresh text and event prediction for assessing LLMs’ temporal adaptability. |
| Outcome: | The proposed framework shows significant temporal biases and a decline in performance over time. |
CaTS-Bench: Can Language Models Describe Time Series? (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing time series captioning benchmarks rely on fully synthetic or generic captions . authors propose a pipeline for generating high-fidelity synthetic captions, which is validated . |
| Approach: | They propose a benchmark for Context-aware Time Series reasoning across 11 diverse domains . they evaluate leading Vision-Language Models on their benchmark . |
| Outcome: | The proposed benchmark evaluates 1746 human-rewritten captions and shows they perform better than open-source models. |
Harnessing LLMs for Temporal Data - A Study on Explainable Financial Time Series Forecasting (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Recent advances in machine learning and artificial intelligence have opened up numerous opportunities and challenges in financial time series forecasting. |
| Approach: | They propose to use Large Language Models for explainable financial time series forecasting to leverage cross-sequence information and extract insights from text and price time series. |
| Outcome: | The proposed model outperforms ARMA-GARCH and gradient-boosting tree models while underperforming on other models. |
Navigating the Modern Evaluation Landscape: Considerations in Benchmarks and Frameworks for Large Language Models (LLMs) (2024.lrec-tutorials)
Copied to clipboard
| Challenge: | General-purpose Language Models have changed the world of Natural Language Processing, if not the world itself. |
| Approach: | This tutorial will lay the foundations and explain the basics of evaluation and compare traditional methods to newly developed methods. |
| Outcome: | The tutorial assumes little familiarity with metrics, datasets, prompts and benchmarks . it will compare traditional methods to newly developed methods . |