Ensembling Large Language Models with Process Reward-Guided Tree Search for Better Complex Reasoning (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for ensembling language models fail to address complex reasoning tasks. |
| Approach: | They propose a framework for process-level ensembling of large language models using Monte Carlo tree search. |
| Outcome: | The proposed framework outperforms both language model decoding and language model ensemble methods on five reasoning benchmarks. |
Similar Papers
DSG-MCTS: A Dynamic Strategy-Guided Monte Carlo Tree Search for Diversified Reasoning in Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown strong potential in complex reasoning tasks, but their performance often degrades, resulting in hallucinations, errors, and logical inconsistencies. |
| Approach: | They propose a framework that integrates multiple reasoning strategies to expand the reasoning space and a dynamic strategy selection mechanism that adapts to the task context. |
| Outcome: | The proposed framework outperforms existing state-of-the-art methods on a set of reasoning benchmarks. |
Towards Hierarchical Multi-Step Reward Models for Enhanced Reasoning in Large Language Models (2026.findings-acl)
Copied to clipboard
Teng Wang, Jiang Zhangyi, Zhenqi He, Hailei Gong, Shenyang Tong, Wenhan Yang, Zeyu Li, Yanan Zheng, Zifan He, Zewen Ye, Shengjie Ma, Jianping Zhang
| Challenge: | Existing Process Reward Models (PRMs) are vulnerable to reward hacking and require expensive, large-scale annotation of reasoning steps. |
| Approach: | They propose a reward model approach which evaluates both individual and consecutive reasoning steps from fine-grained and coarse-grounded level. |
| Outcome: | Empirical results show that the proposed model performs better than existing PRMs and is more robust than existing models. |
ReKG-MCTS: Reinforcing LLM Reasoning on Knowledge Graphs via Training-Free Monte Carlo Tree Search (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to combining knowledge graphs with large language models face limitations in path exploration strategies or excessive computational overhead. |
| Approach: | They propose a training-free framework that synergizes Monte Carlo Tree Search with LLM capabilities to enable dynamic reasoning over KGs. |
| Outcome: | The proposed framework outperforms existing training-free methods and achieves competitive performance compared to fine-tuned baselines. |
From Complex to Simple: Unraveling the Cognitive Tree for Reasoning with Small Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to solve complex logical reasoning problems are cumbersome for language models. |
| Approach: | They propose to use iterative methodology to construct a cognitive tree using language models . they propose to generate multiple responses by utilizing in-context examples . |
| Outcome: | The proposed model achieves a performance level comparable to that of GPT-3.5 . the proposed model contains fewer parameters than 5% of the model with 175B parameters . |
AgentPro: Enhancing LLM Agents with Automated Process Supervision (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing frameworks lack explicit supervision during the reasoning process, which may lead to error propagation across reasoning chains. |
| Approach: | They propose a framework which automates process supervision for large language model agents by automatically generating step-level annotations and developing a process reward model based on these annotations. |
| Outcome: | The proposed framework outperforms existing agent-based methods on four datasets and achieves a 6.32% increase in accuracy. |
Boosting Policy and Process Reward Models with Monte Carlo Tree Search in Open-Domain QA (2025.findings-acl)
Copied to clipboard
Chi-Min Chan, Chunpu Xu, Junqi Zhu, Jiaming Ji, Donghai Hong, Pengcheng Wen, Chunyang Jiang, Zhen Ye, Yaodong Yang, Wei Xue, Sirui Han, Yike Guo
| Challenge: | Experimental results show that our approach can effectively improve the performance of both the policy model and the reward model. |
| Approach: | They propose to use Monte Carlo Tree Search for both policy model improvement and reward model improvement to bridge it to more subtle open-domain question answering. |
| Outcome: | The proposed approach surpasses existing methods for annotation and training data with fewer data points and achieves better performance in test-time scaling strategies. |
Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models (2023.emnlp-main)
Copied to clipboard
Yifan Hou, Jiaoda Li, Yu Fei, Alessandro Stolfo, Wangchunshu Zhou, Guangtao Zeng, Antoine Bosselut, Mrinmaya Sachan
| Challenge: | Recent work has shown that language models (LMs) have strong multi-step (i.e., procedural) reasoning capabilities. |
| Approach: | They propose a mechanistic interpretation of language models for multi-step reasoning tasks by introducing a new probing approach that recovers the reasoning tree from the model’s attention patterns. |
| Outcome: | The proposed model implicitly embeds a reasoning tree resembling the correct reasoning process within it, and detects the information from the model’s attention patterns for most examples. |
MT-RewardTree: A Comprehensive Framework for Advancing LLM-Based Machine Translation via Reward Modeling (2025.findings-emnlp)
Copied to clipboard
| Challenge: | MT-RewardTree provides a framework for constructing, evaluating, and deploying process reward models in machine translation (MT) |
| Approach: | They propose a method for automatically generating token-level preference pairs using approximate Monte Carlo Tree Search. |
| Outcome: | The proposed framework achieves state-of-the-art performance in token-level evaluation and sequence-level analysis. |
LM2: A Simple Society of Language Models Solves Complex Reasoning (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies show that providing guidance via decomposing the original question into multiple subproblems elicits more robustness in LLM reasoning. |
| Approach: | They propose a language-based decomposition, solution and verification framework that modularizes the decomposer, solution, and verification into three different language models. |
| Outcome: | The proposed model outperforms existing methods on in- and out-domain reasoning problems, outperforming the best baselines by 8.1% on MATH, 7.71% on JEEBench, and 9.7% on MedQA problems. |
Monte Carlo Thought Search: Large Language Model Querying for Complex Scientific Reasoning in Catalyst Design (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a goal-driven combinatorial search using large language models has not been explored in detail. |
| Approach: | They propose a Monte Carlo Tree Search-based approach that improves beyond state-of-the-art chain-of thought prompting variants to augment scientific reasoning. |
| Outcome: | The proposed approach improves over the best baseline by 25.8% and can augment scientist’s reasoning and discovery process with novel insights. |