| Challenge: | Recent advances in large language models (LLMs) have enabled them to communicate their confidence in natural language, improving transparency and reliability. |
| Approach: | They propose a framework that promotes answer-grounded confidence estimation and analyze the dynamics of verbalized confidence estimation. |
| Outcome: | The proposed framework significantly improves confidence calibration while exhibiting strong generalization to unseen settings without degrading task performance. |
Similar Papers
Large Language Models Are Overconfident in Their Own Responses (2026.findings-acl)
Copied to clipboard
| Challenge: | Prior work has shown that instruction-tuned large language models are less well calibrated than their base pre-trained counterparts. |
| Approach: | They propose a simple inference-time strategy that frams the model’s answer as user input during confidence elicitation. |
| Outcome: | The proposed approach reduces overconfidence and improves calibration by up to 26% without retraining. |
Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback (2023.emnlp-main)
Copied to clipboard
Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, Christopher Manning
| Challenge: | Recent studies have shown that unsupervised pre-training produces large language models whose conditional probabilities are remarkably well-calibrated. |
| Approach: | They propose to use verbalized confidences to extract confidence from large language models with reinforcement learning from human feedback to improve their accuracy. |
| Outcome: | The proposed methods reduce the expected calibration error by 50% for RLHF-LMs such as ChatGPT, GPT-4, and Claude. |
Confidence Under the Hood: An Investigation into the Confidence-Probability Alignment in Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly used in high-stakes areas such as healthcare, law, and education. |
| Approach: | They propose a concept of Confidence-Probability Alignment that connects an LLM’s internal confidence to the confidence conveyed in the model’s response when explicitly asked about its certainty. |
| Outcome: | The proposed model shows the strongest confidence-probability alignment across a wide range of tasks. |
Do Language Models Mirror Human Confidence? Exploring Psychological Insights to Address Overconfidence in LLMs (2025.findings-acl)
Copied to clipboard
| Challenge: | Psychology research has shown that humans are poor at estimating their performance on tasks, tending towards underconfidence on easy tasks and overconfidence on difficult tasks. |
| Approach: | They propose to use a self-assessment method to assess confidence in large language models (LLMs) they propose to ask for the answer separately and then use them to improve their accuracy. |
| Outcome: | The proposed method improves confidence calibration and interpretability in QA tasks with different personas. |
Confidence Improves Self-Consistency in LLMs (2025.findings-acl)
Copied to clipboard
| Challenge: | Modern large language models (LLMs) demonstrate strong reasoning capabilities, driven in part by their capacity to generate a sequence of intermediate reasoning steps that lead them toward a final answer. |
| Approach: | They propose a method that performs a weighted majority vote based on confidence scores obtained directly from the model. |
| Outcome: | The proposed method outperforms self-consistency on nine models and four datasets, reducing the required number of reasoning paths by over 40% on average. |
Relying on the Unreliable: The Impact of Language Models’ Reluctance to Express Uncertainty (2024.acl-long)
Copied to clipboard
| Challenge: | a pivotal aspect of fostering reliable human-AI interactions lies in the apt communication of model confidences. |
| Approach: | They examine how LMs incorporate confidence in responses via natural language . they also examine how downstream users behave in response to LM-articulated uncertainties . |
| Outcome: | The proposed model overconfidences are high in LMs, and humans are biased against uncertainty-rich texts. |
Calibrating the Confidence of Large Language Models by Eliciting Fidelity (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models with RLHF and RLAIF have good alignment but exhibit overconfidence post-alignment. |
| Approach: | They propose a plug-and-play method to estimate the confidence of large language models. |
| Outcome: | The proposed method has shown good calibration performance on 6 RLHF-LMs on four MCQA datasets. |
Think Just Enough: Leveraging Self-Assessed Confidence for Adaptive Reasoning in Language Models (2026.findings-eacl)
Copied to clipboard
| Challenge: | Recent advances in large reasoning models (LLMs) have shown remarkable capabilities in complex tasks such as mathematical problem solving and code generation. |
| Approach: | They propose a method for optimizing reasoning length via self-assessed confidence. |
| Outcome: | The proposed method improves computational efficiency without compromising answer quality. |
Self-Training Large Language Models with Confident Reasoning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models generate reasoning paths before final answers, but learning such a path requires costly human supervision. |
| Approach: | They propose a method that fine-tunes LLMs to prefer reasoning paths with high confidence . they propose 'cORE-PO' that fine tunes Lms to choose high-quality reasoning paths . |
| Outcome: | The proposed method improves the accuracy of outputs on four in-distribution and two out-of-difference benchmarks. |
All Roads Lead to Rome: Graph-Based Confidence Estimation for Large Language Model Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for confidence estimation are primarily designed for factual QA tasks and fail to generalize to reasoning tasks. |
| Approach: | They propose a set of training-free, graph-based confidence estimation methods tailored to reasoning tasks that exploit graph properties such as centrality, path convergence, and path weighting. |
| Outcome: | The proposed methods improve confidence estimation and performance on two downstream tasks. |