Debjit Paul, Mete Ismayilzada, Maxime Peyrard, Beatriz Borges, Antoine Bosselut, Robert West, Boi Faltings
| Challenge: | Language models (LLMs) have shown remarkable performance by explicitly generating intermediate inferences,e.g., chain-of-thought prompting. |
| Approach: | They propose a framework for finetuning LMs to generate intermediate reasoning steps while interacting with a critic model that provides automated feedback on the reasoning. |
| Outcome: | Empirical evaluations of REFINER on three diverse reasoning tasks show that it significantly improves over baseline models. |
Similar Papers
Invocation Refiner: A Plug-and-Play Module for Rectifying LLM Tool Invocations (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown remarkable capabilities in Tool-Integrated Reasoning (TIR) however, the practical application is often hindered by frequent errors in tool invocations, such as incorrect tool names, invalid parameters, wrong tool-call order, or malformed invocation formats. |
| Approach: | They propose a specialized post-processing module that performs independent reasoning on the input of a frozen upstream LLM and an advanced RL algorithm to improve the tool-use reliability of base LLMs. |
| Outcome: | The proposed module improves task completion rates and invocation accuracy over the raw outputs of various upstream LLMs on a diverse set of tool-use and reasoning benchmarks. |
CriticBench: Benchmarking LLMs for Critique-Correct Reasoning (2024.findings-acl)
Copied to clipboard
| Challenge: | CriticBench is a benchmark designed to assess LLMs’ abilities to critique and refine their reasoning across a variety of tasks. |
| Approach: | They propose a benchmark to assess LLMs' ability to critique and correct reasoning across a variety of tasks. |
| Outcome: | The proposed benchmark examines the performance of 17 large language models in generation, critique, and correction reasoning. |
Learning to Refine with Fine-Grained Natural Language Feedback (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work has explored the capability of large language models to identify and correct errors in LLM-generated responses. |
| Approach: | They propose to combine refinement with feedback into three distinct competencies . step 1: Detect, Critique, Refine gives a fine-grained feedback about errors . |
| Outcome: | The proposed method outperforms existing refinement approaches and models not fine-tuned for factuality critiquing. |
ReFT: Reasoning with Reinforced Fine-Tuning (2024.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to improve the generalization of large language models are using Supervised Fine-Tuning (SFT) this approach does not show sufficient generalization ability because it only relies on the given CoT data. |
| Approach: | They propose to use Chain-of-Thought annotations to train Large Language Models using supervised fine-tuning to improve generalization. |
| Outcome: | The proposed approach outperforms SFT on GSM8K, MathQA, and SVAMP datasets and shows a superior generalization ability. |
Can Large Language Models Detect Errors in Long Chain-of-Thought Reasoning? (2025.acl-long)
Copied to clipboard
Yancheng He, Shilong Li, Jiaheng Liu, Weixun Wang, Xingyuan Bu, Ge Zhang, Z.y. Peng, Zhaoxiang Zhang, Zhicheng Zheng, Wenbo Su, Bo Zheng
| Challenge: | Recent advances in o1-like models have generated long Chain-of-Thought reasoning steps to improve the reasoning abilities of existing Large Language Models (LLMs). |
| Approach: | They propose a DeltaBench to analyze the quality and effectiveness of o1-like models and measure their ability to detect errors in long COT reasoning. |
| Outcome: | The proposed model can detect errors in long COT reasoning. |
LTRAG: Enhancing Autoformalization and Self-refinement for Logical Reasoning with Thought-Guided RAG (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown promise in natural language reasoning, especially with techniques like chain-of-thought prompting. |
| Approach: | They propose a framework to enhance autoformalization and self-refinement for logical reasoning with Retrieval-Augmented Generation (RAG) by building knowledge bases of thought-guided examples. |
| Outcome: | The proposed framework outperforms Logic-LM and LINC on FOLIO and AR-LSAT, and achieves an accuracy gain of 13% over Logic LM and the proposed methods on GPT-4 and AR LSAT. |
MAF: Multi-Aspect Feedback for Improving Reasoning in Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to enhance Language Models fail to address diverse error types . generic feedback is a bottleneck for addressing diverse errors in reasoning chains . |
| Approach: | They propose an iterative refinement framework that integrates multiple feedback modules . they propose to address errors in reasoning chains by integrating frozen LMs with external tools . |
| Outcome: | The proposed framework improves performance in Mathematical Reasoning and Logical Entailment by 20% and 18% respectively. |
Critic-CoT: Boosting the Reasoning Abilities of Large Language Model via Chain-of-Thought Critic (2025.findings-acl)
Copied to clipboard
Xin Zheng, Jie Lou, Boxi Cao, Xueru Wen, Yuqiu Ji, Hongyu Lin, Yaojie Lu, Xianpei Han, Debing Zhang, Le Sun
| Challenge: | Existing approaches to improve the reasoning performance of large language models rely on intuitive instance-level feedback, which limits the reasoning capabilities. |
| Approach: | They propose a framework that pushes LLMs toward System-2-like critic capability by using a step-wise CoT reasoning paradigm and automatic construction of weak-supervision data without human annotation. |
| Outcome: | The proposed model significantly improves task-solving performance by filtering out invalid solutions or iterative refinement. |
CriticLean: Critic-Guided Reinforcement Learning for Mathematical Formalization (2026.acl-long)
Copied to clipboard
Zhongyuan Peng, Yifan Yao, Kaijing Ma, Shuyue Guo, Yizhe Li, Yichi Zhang, Chenchen Zhang, Yifan Zhang, Zhouliang Yu, Luming Li, Minghao Liu, Yihang Xia, Jiawei Shen, Yuchen Wu, Yixin Cao, Zhaoxiang Zhang, Wenhao Huang, Jiaheng Liu, Ge Zhang
| Challenge: | Existing approaches to formalizing mathematical statements face limitations in accuracy, especially in the context of complex, highlevel problems that involve sophisticated mathematical reasoning. |
| Approach: | They propose a CriticLean framework that elevates the role of the critic from a passive validator to an active learning component and introduce a benchmark to measure models’ ability to distinguish semantically correct from incorrect formalizations. |
| Outcome: | The proposed framework outperforms open- and closed-source benchmarks and shows that it significantly outperformed existing models. |
Small Language Models Need Strong Verifiers to Self-Correct Reasoning (2024.findings-acl)
Copied to clipboard
Yunxiang Zhang, Muhammad Khalifa, Lajanugen Logeswaran, Jaekyeom Kim, Moontae Lee, Honglak Lee, Lu Wang
| Challenge: | Existing studies show that large language models can self-correct their outputs by generating a critique and revising it based on the critique. |
| Approach: | They propose a pipeline that prompts small language models to collect self-correction data that supports the training of self-refinement abilities. |
| Outcome: | The proposed pipeline improves the self-correction abilities of two models on five datasets spanning math and commonsense reasoning. |