Unlocking Recursive Thinking of LLMs: Alignment via Refinement (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for recursive reasoning are limited due to lack of expert-curated data. |
| Approach: | They propose a method that unlocks the potential of Large Language Models for recursive reasoning through long-form Chain of Thought. |
| Outcome: | The proposed method outperforms preference optimization methods on the openAI o1-series models by 20% on 3k synthetic samples. |
Similar Papers
The ART of LLM Refinement: Ask, Refine, and Trust (2024.naacl-long)
Copied to clipboard
Kumar Shridhar, Koustuv Sinha, Andrew Cohen, Tianlu Wang, Ping Yu, Ramakanth Pasunuru, Mrinmaya Sachan, Jason Weston, Asli Celikyilmaz
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable generative abilities, but can they judge the quality of their own generations and self-improve? |
| Approach: | They propose a reasoning with a refinement strategy called *ART: Ask, Refine, and Trust* that asks necessary questions to decide when an LLM should refine its output and uses it to affirm or deny trust. |
| Outcome: | The proposed reasoning with a refinement strategy achieves a performance gain of +5 points over baselines on two multistep reasoning tasks. |
Correct, Concise and Complete: Multi-stage Training For Adaptive Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) increase test-time computation, often in the form of chain-of-thought (CoT) however, reasoning traces can become unnecessarily long, increasing computation costs without improving accuracy and sometimes even degrading performance. |
| Approach: | They propose a multi-stage efficient reasoning method that combines supervised fine-tuning with reinforcement learning using an adaptive length penalty. |
| Outcome: | The proposed method reduces response length by an average of 28% for 8B models and 40% for 32B models while incurring only minor performance drops of 1.6 and 2.5 points, respectively. |
Improving Preference Alignment of LLM with Inference-Free Self-Refinement (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) develop in-context learning capability through pretraining and instruction tuning. |
| Approach: | Large language models (LLMs) develop in-context learning capability through pretraining and instruction tuning. |
| Outcome: | Experiments show that incorporating IFSR into preference alignment yields performance improvement over 10%. |
Learning to Refine: Self-Refinement of Parallel Reasoning in LLMs (2026.findings-acl)
Copied to clipboard
Qibin Wang, Pu Zhao, Shaohan Huang, Fangkai Yang, Lu Wang, Furu Wei, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang
| Challenge: | Existing approaches to test-time scaling are limited due to the quality of candidate responses. |
| Approach: | They propose a new metric to quantify the relative improvement of self-refinement beyond majority voting. |
| Outcome: | The proposed method achieves state-of-the-art performance across five benchmarks over other methods. |
Enhancing Reasoning Abilities of Small LLMs with Cognitive Alignment (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to distill chain-of-thought (CoT) results from large language reasoning models (LRMs) to small models are ineffective and require substantial amount of annotated data. |
| Approach: | They propose a Critique-Rethink-Verify system for training small language reasoning models that can be critiquized according to the cognitive capabilities of smaller models. |
| Outcome: | The proposed system outperforms other methods on challenging reasoning benchmarks. |
Fine-Tuning on Diverse Reasoning Chains Drives Within-Inference CoT Refinement in LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to generate multiple independent CoTs, combining them through ensembling or other post-hoc strategies, have been shown to be effective in boosting performance. |
| Approach: | They propose a method where LLMs are fine-tuned to generate a sequence of Diverse Chains of Thought (DCoT) within a single inference step. |
| Outcome: | The proposed model can generate multiple chains of thought within a single inference step without external feedback. |
Too Long, Do Re-weighting for Efficient LLM Reasoning Compression (2026.acl-long)
Copied to clipboard
Zhong-Zhi Li, Xiao Liang, Zihao Tang, Lei Ji, Peijie Wang, Haotian Xu, Xing W, Haizhen Huang, Weiwei Deng, Yeyun Gong, Zhijiang Guo, Xiao Liu, Fei Yin, Cheng-Lin Liu
| Challenge: | Large Language Models (LLMs) have recently achieved remarkable progress on complex reasoning tasks by leveraging extended Chain-of-Thought (CoT) techniques. |
| Approach: | They propose a method that uses Extended Chain-of-Thought (EFT) to reduce the number of output tokens by nearly 40% while maintaining the accuracy of the reasoning. |
| Outcome: | The proposed method reduces the number of output tokens by nearly 40% while maintaining the accuracy of the reasoning. |
Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judge (2025.emnlp-main)
Copied to clipboard
Tianhao Wu, Weizhe Yuan, Olga Golovneva, Jing Xu, Yuandong Tian, Jiantao Jiao, Jason E Weston, Sainbayar Sukhbaatar
| Challenge: | Existing methods for improving large language models have focused on improving model responses rather than judgment capabilities, resulting in rapid saturation during iterative training. |
| Approach: | They propose an iterative Meta-Rewarding step where the model judges its own judgements and uses that feedback to refine its judgment skills. |
| Outcome: | The proposed model improves Llama-3-8B-Instruct from 22.9% to 39.4% on AlpacaEval 2 and 20.6% to 29.1% on Arena-Hard. |
Exploring Chain-of-Thought Reasoning for Steerable Pluralistic Alignment (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are typically trained to reflect a relatively uniform set of values, which limits their applicability to tasks that require understanding of nuanced human perspectives. |
| Approach: | They propose to use Chain-of-Thought reasoning techniques to build steerable pluralistic models by fine-tuning on human-authored CoT and synthetic explanations. |
| Outcome: | The proposed methods outperform others and demonstrate strong sample efficiency. |
User Feedback Alignment for LLM-powered Exploration in Large-scale Recommendation Systems (2025.acl-industry)
Copied to clipboard
Jianling Wang, Yifan Liu, Yinghao Sun, Xuejian Ma, Yueqi Wang, He Ma, Zhengyang Su, Minmin Chen, Mingyan Gao, Onkar Dalal, Ed H. Chi, Lichan Hong, Ningren Han, Haokai Lu
| Challenge: | Large Language Models (LLMs) can be used to broaden user experiences beyond established preferences and reinforce feedback loops. |
| Approach: | They propose a hierarchical approach that combines hierarchic planning with LLM inference-time scaling to improve recommendation relevancy without compromising novelty. |
| Outcome: | The proposed approach shows significant gains in both user satisfaction and exploration diversity. |