Self-Training Large Language Models with Confident Reasoning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models generate reasoning paths before final answers, but learning such a path requires costly human supervision. |
| Approach: | They propose a method that fine-tunes LLMs to prefer reasoning paths with high confidence . they propose 'cORE-PO' that fine tunes Lms to choose high-quality reasoning paths . |
| Outcome: | The proposed method improves the accuracy of outputs on four in-distribution and two out-of-difference benchmarks. |
Similar Papers
SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationales (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to elicit confidence from large language models are limited to binary or inaccurate group-level confidence estimates. |
| Approach: | They propose a training framework that teaches LLMs to express more fine-grained confidence estimates. |
| Outcome: | The proposed training framework reduces the confidence calibration error and maintains the performance of the model. |
Confidence Improves Self-Consistency in LLMs (2025.findings-acl)
Copied to clipboard
| Challenge: | Modern large language models (LLMs) demonstrate strong reasoning capabilities, driven in part by their capacity to generate a sequence of intermediate reasoning steps that lead them toward a final answer. |
| Approach: | They propose a method that performs a weighted majority vote based on confidence scores obtained directly from the model. |
| Outcome: | The proposed method outperforms self-consistency on nine models and four datasets, reducing the required number of reasoning paths by over 40% on average. |
Large Language Models Can Self-Improve (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have excellent performance in various tasks, but fine-tuning requires extensive supervision. |
| Approach: | They propose to use a pre-trained Large Language Model to generate rationale-augmented answers for unlabeled questions and fine-tune the LLM using those self-generated solutions as target outputs. |
| Outcome: | The proposed approach improves the general reasoning ability of a 540B-parameter LLM without any ground truth label. |
Self-Training Elicits Concise Reasoning in Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Chain-of-thought reasoning has enabled large language models to use additional computation through intermediate tokens to solve complex tasks, but current models often generate more tokens than necessary to accomplish the task, incurring extraneous inference costs. |
| Approach: | They propose to fine-tune models with self-generated concise reasoning paths obtained by best-of-N sampling and few-shot conditioning in task-specific settings to elicit concise reasoning. |
| Outcome: | The proposed method reduces output tokens by 30% on GSM8K and MATH while maintaining average accuracy. |
All Roads Lead to Rome: Graph-Based Confidence Estimation for Large Language Model Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for confidence estimation are primarily designed for factual QA tasks and fail to generalize to reasoning tasks. |
| Approach: | They propose a set of training-free, graph-based confidence estimation methods tailored to reasoning tasks that exploit graph properties such as centrality, path convergence, and path weighting. |
| Outcome: | The proposed methods improve confidence estimation and performance on two downstream tasks. |
Beyond Correctness: Confidence-Aware Reward Modeling for Enhancing Large Language Model Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large language models have shifted the post-training paradigm from instruction tuning and human preference alignment to reinforcement learning (RL) based on rule-based evaluations of answer correctness, these models often receive rewards for speculative answers without generating coherent reasoning chains. |
| Approach: | They propose a confidence-based reward model tailored for enhancing STEM reasoning capabilities. |
| Outcome: | The proposed model outperforms state-of-the-art open-source reward models across diverse STEM benchmarks. |
Self-Ensemble: Mitigating Confidence Distortion for Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models exhibit a confidence distortion problem on multichoice question-answering . Self-Ensemble solves this problem by splitting the choices into several groups . |
| Approach: | They propose a method that splits LLM choices into several groups and ensembles them to reach a final decision. |
| Outcome: | The proposed method outperforms standard inference and baseline methods on MCQA. |
Self-training Large Language Models through Knowledge Detection (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) often require extensive labeled datasets and training compute to achieve impressive performance across downstream tasks. |
| Approach: | They propose a self-training paradigm where the LLM curates its own labels and selectively trains on unknown data samples identified through a reference-free consistency method. |
| Outcome: | The proposed model reduces the dependency on large labeled datasets and mitigates catastrophic forgetting in out-of-distribution benchmarks. |
Making Pre-trained Language Models both Task-solvers and Self-calibrators (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work shows that pre-trained language models can be effective for high-stake applications, but they become overconfident in their wrong predictions. |
| Approach: | They propose to use extra data to train pre-trained language models to effectively utilize training samples to make them both task-solvers and self-calibrators. |
| Outcome: | The proposed method can be used in three downstream applications, including selective classification, adversarial defense, and model cascading. |
Self-Training with Direct Preference Optimization Improves Chain-of-Thought Reasoning (2024.acl-long)
Copied to clipboard
| Challenge: | Recent studies focus on enhancing large-scale language models' reasoning abilities, but the research question of how to GSM8K Performance vs. computational cost remains. |
| Approach: | They propose to train small-scale language models with their own outputs to avoid relying on large models' outputs. |
| Outcome: | The proposed approach outperforms baseline models with comparable sizes while minimizing the required compute. |