| Challenge: | Existing PRMs cannot capture richer dimensions of information-seeking steps, such as tool interactions and reasoning over tool outputs. |
| Approach: | They propose a generative PRM trained with dual capabilities that compresses the growing context while preserving essential information for step evaluation. |
| Outcome: | PRInTS improves on FRAMES, GAIA, and WebWalkerQA models while preserving essential information for step evaluation. |
Similar Papers
A Comprehensive Survey of Process Reward Models: Data Generation, Model Construction, and Usage (2026.acl-long)
Copied to clipboard
Congmin Zheng, Jiachen Zhu, Zhuoying Ou, Yuxiang Chen, Kangning Zhang, Rong Shan, Zeyu Zheng, Mengyue Yang, Jianghao Lin, Yong Yu, Weinan Zhang
| Challenge: | Large Language Models (LLMs) have advanced reasoning ability, yet conventional alignment remains dominated by outcome reward models that judge only final answers. |
| Approach: | They summarize applications across math, code, text, multimodal reasoning, robotics, and agents . goal is to clarify design spaces, reveal open challenges, and guide future research toward fine-grained, robust reasoning alignment. |
| Outcome: | The proposed model enables finer credit assignment, richer diagnostics, and improved robustness. |
ToolPRMBench: Evaluating and Advancing Process Reward Models for Tool-using Agents (2026.findings-acl)
Copied to clipboard
| Challenge: | Reward-guided search methods have shown potential in enhancing tool-using agents . however, there is a lack of reliable evaluation benchmarks for PRMs in tool-use settings . |
| Approach: | They propose a large-scale benchmark specifically designed to evaluate PRMs for tool-using agents. |
| Outcome: | The proposed benchmark shows that tool reward models perform better in tool-using environments. |
R-PRM: Reasoning-Driven Process Reward Modeling (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing Process Reward Models (PRMs) output evaluation scores directly, limiting both learning efficiency and evaluation accuracy. |
| Approach: | They propose a Reasoning-Driven Process Reward Modeling (R-PRM) which activates inherent reasoning to enhance process-level evaluation. |
| Outcome: | The proposed model outperforms baseline models on ProcessBench and PRMBench by 13.9 and 8.5 F1 scores. |
The Lessons of Developing Process Reward Models in Mathematical Reasoning (2025.findings-acl)
Copied to clipboard
Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, Bowen Yu, Dayiheng Liu, Jingren Zhou, Junyang Lin
| Challenge: | a recent study shows that process reward models can make mistakes, leading to wrong conclusions. |
| Approach: | They propose a consensus filtering mechanism that integrates MC estimation with LLM-as-a-judge to improve model performance and data efficiency. |
| Outcome: | The proposed model outperforms existing open-source alternatives and provides practical guidelines for future research. |
Process-Supervised Reward Models for Verifying Clinical Note Generation: A Scalable Approach Guided by Domain Expertise (2025.emnlp-main)
Copied to clipboard
Hanyin Wang, Chufan Gao, Qiping Xu, Bolun Liu, Guleid Hussein, Hariprasad Reddy Korsapati, Mohamad El Labban, Kingsley Iheasirim, Mohamed Hassan, Gokhan Anil, Brian Bartlett, Jimeng Sun
| Challenge: | Currently, no automated, scalable method exists to evaluate the quality of LLM-generated clinical notes, leaving manual evaluation the gold standard. |
| Approach: | They propose a framework for training PRMs to deliver step-level reward signals for LLM-generated clinical notes. |
| Outcome: | The proposed framework outperforms reasoning and non-reasoning models on key evaluations and selects physician-preferred clinical notes with 56.2% accuracy. |
Process Reward Models Meet Planning: Generating Precise and Scalable Datasets for Step-Level Rewards (2026.acl-long)
Copied to clipboard
| Challenge: | Existing PRM datasets are expensive to construct and limited to the mathematical domain. |
| Approach: | They propose a method to generate a corpus of one million reasoning steps using the Planning Domain Definition Language. |
| Outcome: | The proposed model generates a corpus of approximately one million reasoning steps across various PDDL domains and trains them. |
Retrieval-Augmented Process Reward Model for Generalizable Mathematical Reasoning (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have advanced mathematical reasoning, but they still struggle with out-of-distribution (OOD) issues. |
| Approach: | They propose a framework to evaluate the logical validity of reasoning steps . they retrieves semantically similar questions and steps for PRM as a warmup . |
| Outcome: | The proposed framework outperforms baseline models on multiple real-world datasets. |
Exploring Generative Process Reward Modeling for Semi-Structured Data: A Case Study of Table Question Answering (2026.eacl-short)
Copied to clipboard
| Challenge: | Recent advances in process reward models (PRMs) have demonstrated remarkable improvements in the reasoning capabilities of large language models (LLMs). |
| Approach: | They evaluate state-of-the-art generative PRMs on table question answering from answer and step perspectives and compare their results to previous studies. |
| Outcome: | The proposed model can aid solution selection but struggle to generalize to out-of-domain data. |
Error Typing for Smarter Rewards: Improving Process Reward Models with Error-Aware Hierarchical Supervision (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are prone to hallucination, especially during multihop tasks. |
| Approach: | They propose a hierarchical, erroraware discriminative PRM that classifies math errors at each step and combines finegrained signals to estimate step correctness. |
| Outcome: | The proposed model outperforms the prior best in a new stateof-theart PRMScore of 67.7 on a 400Ksample dataset . |
ViLBench: A Suite for Vision-Language Process Reward Modeling (2025.emnlp-main)
Copied to clipboard
| Challenge: | Despite its advantages, evaluation on PRMs remains less explored, especially in the multimodal domain. |
| Approach: | They propose to benchmark vision large language models as output reward models and process reward models as process-supervised reward models. |
| Outcome: | The proposed model outperforms both ORM and PRM on vision-language benchmarks and achieves an average improvement of 3.3% over standard CoT and up to 2.5% over its untrained counterpart on ViLBench. |