Genius: A Generalizable and Purely Unsupervised Self-Training Framework For Advanced Reasoning (2025.acl-long)
Copied to clipboard
Fangzhi Xu, Hang Yan, Chang Ma, Haiteng Zhao, Qiushi Sun, Kanzhi Cheng, Junxian He, Jun Liu, Zhiyong Wu
| Challenge: | Existing methods for enhancing LLM reasoning rely on supervisory signals . current methods rely heavily on outcome supervision and auxiliary reward models . |
| Approach: | They propose a gen-eralizable and purely unsupervised self-training framework to enhance LLM reasoning without supervision. |
| Outcome: | The proposed framework improves LLM reasoning without supervision without external supervision. |
Similar Papers
StepSearch: Igniting LLMs Search Ability via Step-Wise Proximal Policy Optimization (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent work has demonstrated unprecedented capabilities in sophisticated linguistic comprehension and generative tasks. |
| Approach: | They propose a framework for search LLMs that trains with step-wise proximal policy optimization method to improve QA performance. |
| Outcome: | The proposed framework outperforms global-reward benchmarks on multi-hop QA with a stepwise proximal policy optimization method and richer and more detailed intermediate search rewards and token-level process supervision. |
Mitigating Tail Narrowing in LLM Self-Improvement via Socratic-Guided Sampling (2025.naacl-long)
Copied to clipboard
Yiwen Ding, Zhiheng Xi, Wei He, Lizhuoyuan Lizhuoyuan, Yitao Zhai, Shi Xiaowei, Xunliang Cai, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Large language models (LLMs) generate solutions themselves and iteratively train on filtered, high-quality rationales, but performance reaches a ceiling after a few iterations. |
| Approach: | They propose a strategy to improve the efficiency of sampling heavy-tailed data by using Socratic-style guidance signals to help LLMs reasoning with complex queries. |
| Outcome: | The proposed approach is effective on difficult queries and on held-out tasks, while requiring human supervision. |
Teaching LLMs to Plan, Not Just Solve: Plan Learning Boosts LLMs Generalization in Reasoning Tasks (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for reinforcement learning (RL) on self-generated data are limited in many domains. |
| Approach: | a new framework combines plan-based search with Step-level Advantage Preference Optimization to optimize plan learning. |
| Outcome: | The proposed framework improves in-domain performance and out-of-domain benchmarks. |
Current Advances in LLM Reasoning (2026.acl-tutorials)
Copied to clipboard
| Challenge: | This tutorial examines comprehensive evaluation strategies to assess the reasoning abilities of large language models (LLMs) advanced inference time methods and post-training methods that aim to make LLMs think more like humans are discussed in this tutorial. |
| Approach: | This tutorial explores comprehensive evaluation strategies to assess the reasoning abilities of large language models (LLMs) and discusses two types of methods to improve models’ reasoning: advanced inference time methods, structured and self-improvement inference methods, and post-training methods, such as RLHF, DPO, and GRPO. |
| Outcome: | This tutorial examines evaluation strategies to assess the reasoning abilities of large language models and discusses two types of methods to improve models’ reasoning. |
EpiCaR: Knowing What You Don’t Know Matters for Better Reasoning in LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to improving reasoning abilities of large language models incur a significant calibration cost. |
| Approach: | They propose an epistemic learning problem that integrates reasoning and calibration into an iterative supervised training framework. |
| Outcome: | The proposed method achieves Pareto-superiority over standard baselines in accuracy and calibration. |
Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judge (2025.emnlp-main)
Copied to clipboard
Tianhao Wu, Weizhe Yuan, Olga Golovneva, Jing Xu, Yuandong Tian, Jiantao Jiao, Jason E Weston, Sainbayar Sukhbaatar
| Challenge: | Existing methods for improving large language models have focused on improving model responses rather than judgment capabilities, resulting in rapid saturation during iterative training. |
| Approach: | They propose an iterative Meta-Rewarding step where the model judges its own judgements and uses that feedback to refine its judgment skills. |
| Outcome: | The proposed model improves Llama-3-8B-Instruct from 22.9% to 39.4% on AlpacaEval 2 and 20.6% to 29.1% on Arena-Hard. |
An Efficient and Precise Training Data Construction Framework for Process-supervised Reward Model in Mathematical Reasoning (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for constructing process supervision training data are costly or suffer from poor quality. |
| Approach: | They propose a framework called EpicPRM which annotates each intermediate reasoning step based on its quantified contribution and uses an adaptive binary search algorithm to enhance annotation precision and efficiency. |
| Outcome: | The proposed framework improves annotation precision and efficiency and can be used to train a high-quality training dataset with 50k annotated intermediate steps. |
RATIONALYST: Pre-training Process-Supervision for Improving Reasoning (2025.acl-long)
Copied to clipboard
Dongwei Jiang, Guoxuan Wang, Yining Lu, Andrew Wang, Jingyu Zhang, Chuyu Liu, Benjamin Van Durme, Daniel Khashabi
| Challenge: | RATIONALYST is a model for process-supervision of reasoning based on pretraining on rationale annotations extracted from unlabeled data. |
| Approach: | They propose a model for process-supervision of reasoning based on pre-training on rationale annotations extracted from unlabeled data. |
| Outcome: | RATIONALYST improves reasoning accuracy by 3.9% on representative reasoning benchmarks. |
Self-Training Large Language Models with Confident Reasoning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models generate reasoning paths before final answers, but learning such a path requires costly human supervision. |
| Approach: | They propose a method that fine-tunes LLMs to prefer reasoning paths with high confidence . they propose 'cORE-PO' that fine tunes Lms to choose high-quality reasoning paths . |
| Outcome: | The proposed method improves the accuracy of outputs on four in-distribution and two out-of-difference benchmarks. |
Large Language Models Can Self-Improve (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have excellent performance in various tasks, but fine-tuning requires extensive supervision. |
| Approach: | They propose to use a pre-trained Large Language Model to generate rationale-augmented answers for unlabeled questions and fine-tune the LLM using those self-generated solutions as target outputs. |
| Outcome: | The proposed approach improves the general reasoning ability of a 540B-parameter LLM without any ground truth label. |