Confidence over Time: Confidence Calibration with Temporal Logic for Large Language Model Reasoning

Challenge:	Existing confidence estimation methods reduce reasoning process to a single scalar score, ignoring how confidence evolves throughout generation.
Approach:	They propose to characterize the stepwise confidence signal using Signal Temporal Logic (STL) based on a discriminative STL mining procedure, they find temporal formulas that distinguish correct and incorrect responses.
Outcome:	The proposed method can distinguish between correct and incorrect reasoning signals.

Similar Papers

Overview

Challenge

Approach

Outcome

All Roads Lead to Rome: Graph-Based Confidence Estimation for Large Language Model Reasoning (2025.emnlp-main)

Copied to clipboard

Caiqi Zhang, Chang Shu, Ehsan Shareghi, Nigel Collier

Semantic Text Processing

Challenge:	Existing methods for confidence estimation are primarily designed for factual QA tasks and fail to generalize to reasoning tasks.
Approach:	They propose a set of training-free, graph-based confidence estimation methods tailored to reasoning tasks that exploit graph properties such as centrality, path convergence, and path weighting.
Outcome:	The proposed methods improve confidence estimation and performance on two downstream tasks.

A Survey of Confidence Estimation and Calibration in Large Language Models (2024.naacl-long)

Copied to clipboard

Jiahui Geng, Fengyu Cai, Yuxia Wang, Heinz Koeppl, Preslav Nakov, Iryna Gurevych

Language Models

Challenge:	Large language models (LLMs) have demonstrated impressive capabilities across a wide range of tasks in various domains, but they can be unreliable due to factual errors in their generations.
Approach:	They summarize recent advances in LLM confidence estimation and calibration and outline their main lessons learned.
Outcome:	The proposed methods can be used to assess the reliability of models and to calibrate them across tasks.

How Reliable are Confidence Estimators for Large Reasoning Models? A Systematic Benchmark on High-Stakes Domains (2026.eacl-long)

Copied to clipboard

Reza Khanmohammadi, Erfan Miahi, Simerjot Kaur, Charese Smiley, Ivan Brugere, Kundan S Thind, Mohammad M. Ghassemi

Reasoning

Challenge:	Large Reasoning Models often struggle with confidence calibration, authors say . authors: accurate confidence scores are essential to build trustworthy systems .
Approach:	They propose a Reasoning Model Confidence estimation benchmark to assess LRM confidence . the benchmark is constructed from 347,496 reasoning traces from six popular LRMs .
Outcome:	The proposed benchmark compares ten different representation-based methods on a wide range of architectures.

DateLogicQA: Benchmarking Temporal Biases in Large Language Models (2025.naacl-srw)

Copied to clipboard

Gagan Bhatia, Ming Ze Tang, Cristina Mahanta, Madiha Kazi

Language Models

Challenge:	DateLogicQA examines temporal biases in Large Language Models (LLMs) 190 questions are curated by humans to examine temporal reasoning across date formats and contexts .
Approach:	They propose a human-curated benchmark of 190 questions specifically designed to understand temporal bias in Large Language Models.
Outcome:	The proposed dataset covers seven date formats across past, present, and future contexts . it examines four reasoning types: commonsense, factual, conceptual, and numerical .

Confidence-Driven Multi-Scale Model Selection for Cost-Efficient Inference (2026.findings-eacl)

Copied to clipboard

Bo-Wei Chen, Chung-Chi Chen, An-Zi Yen

Green & Sustainable NLP

Challenge:	Large Language Models (LLMs) have revolutionized inference across diverse natural language tasks, with larger models performing better but at higher computational costs.
Approach:	They propose a confidence-driven strategy that dynamically selects the most suitable model based on confidence estimates.
Outcome:	The proposed approach reduces token usage by approximately 60% and improves cost efficiency on the Massive Multitask Language Understanding (MMLU) benchmark.

Think Just Enough: Leveraging Self-Assessed Confidence for Adaptive Reasoning in Language Models (2026.findings-eacl)

Copied to clipboard

Junyeob Kim, Sang-goo Lee, Taeuk Kim

Language Models

Challenge:	Recent advances in large reasoning models (LLMs) have shown remarkable capabilities in complex tasks such as mathematical problem solving and code generation.
Approach:	They propose a method for optimizing reasoning length via self-assessed confidence.
Outcome:	The proposed method improves computational efficiency without compromising answer quality.

Do Language Models Have a Common Sense regarding Time? Revisiting Temporal Commonsense Reasoning in the Era of Large Language Models (2023.emnlp-main)

Copied to clipboard

Raghav Jain, Daivik Sojitra, Arkadeep Acharya, Sriparna Saha, Adam Jatowt, Sandipan Dandapat

Reasoning

Challenge:	Temporal reasoning is a vital component of human communication and understanding, yet remains an underexplored area within the context of Large Language Models (LLMs).
Approach:	They propose to use 3 prompting strategies to evaluate 8 different LLMs across 6 datasets and 2 Code Generation LMs to perform the analysis.
Outcome:	The proposed models perform better on NLP tasks than the standard models on the same dataset.

AdapTime: Enabling Adaptive Temporal Reasoning in Large Language Models (2026.findings-acl)

Copied to clipboard

Yimin Deng, Yejing Wang, Zhenxi Lin, Zichuan Fu, Guoshuai Zhao, Derong Xu, Yefeng Zheng, Xiangyu Zhao, Xian Wu, Li Zhu, Xueming Qian

Language Models

Challenge:	Existing methods for temporal reasoning are limited and apply a fixed pipeline to all questions.
Approach:	They propose an adaptive temporal reasoning method that dynamically executes reasoning steps based on context and task requirements.
Outcome:	Experiments on two temporal QA benchmarks show the proposed method works.

Confidence Leaps in LLM Reasoning: Early Stopping and Cross-Model Transfer (2026.eacl-short)

Copied to clipboard

Pavel Tikhonov, Ivan Oseledets, Elena Tutubalina

Reasoning

Challenge:	Large Language Models build confidence gradually during reasoning, but internal dynamics of how confidence evolves during this reasoning process remain poorly understood.
Approach:	They propose a model-agnostic early-stopping heuristic that halts generation upon detecting a "confidence leap" they argue that conviction is often reached in a discrete "moment of insight" they propose to train models without sacrificing accuracy .
Outcome:	The proposed model-agnostic heuristic reduces generation time without sacrificing accuracy and significantly reduces the generation time.

Towards Benchmarking and Improving the Temporal Reasoning Capability of Large Language Models (2023.acl-long)

Copied to clipboard

Qingyu Tan, Hwee Tou Ng, Lidong Bing

Language Models

Challenge:	Recent time-dependent question answering datasets tend to be biased in either their coverage of time spans or question types.
Approach:	They propose a temporal reasoning framework based on temporal span extraction and time-sensitive reinforcement learning to improve the temporal ability of large language models.
Outcome:	The proposed framework improves the temporal reasoning capability of large language models by using temporal span extraction and time-sensitive reinforcement learning.