Papers with critic
REFINER: Reasoning Feedback on Intermediate Representations (2024.eacl-long)
Copied to clipboard
Debjit Paul, Mete Ismayilzada, Maxime Peyrard, Beatriz Borges, Antoine Bosselut, Robert West, Boi Faltings
| Challenge: | Language models (LLMs) have shown remarkable performance by explicitly generating intermediate inferences,e.g., chain-of-thought prompting. |
| Approach: | They propose a framework for finetuning LMs to generate intermediate reasoning steps while interacting with a critic model that provides automated feedback on the reasoning. |
| Outcome: | Empirical evaluations of REFINER on three diverse reasoning tasks show that it significantly improves over baseline models. |
Cross-Refine: Improving Natural Language Explanation Generation by Learning in Tandem (2025.coling-main)
Copied to clipboard
| Challenge: | Natural language explanations (NLEs) are vital for elucidating the reasoning behind large language model (LLM) decisions. |
| Approach: | They propose a role-modeling approach that employs two LLMs as generator and critic to generate and refine NLEs. |
| Outcome: | The proposed model outperforms self-refine and can perform with less powerful LLMs. |
BV-Blend: Uncertainty-Weighted Historical Baselines for Stable Critic-Free RL with Verifiable Rewards (2026.findings-acl)
Copied to clipboard
| Challenge: | Critic-free reinforcement learning with verifiable rewards (RLVR) is a practical paradigm for aligning Large Language Models. |
| Approach: | They propose a framework that stabilizes advantage estimation by combining prompt-local on-policy statistics with semantic-cluster-conditioned historical moments. |
| Outcome: | Experiments show that RLVR improves training stability and performance compared to critic-based methods . compared with other approaches, RL VR improves in cold-start regimes with binary verifiers . |
Learning Selective LLM Autonomy from Copilot Feedback in Enterprise Customer Support Workflows (2026.acl-industry)
Copied to clipboard
Nikita Borovkov, Elisei Rykov, Olga Tsymboi, Sergei Filimonov, Nikita Surnachev, Dmitry Bitman, Anatolii Potapov
| Challenge: | 45% of sessions automated in production without degrading support quality level . traditional automated processes are costly at scale and require manual rule authoring . |
| Approach: | They propose a system that automates end-to-end customer support workflows inside an enterprise BPM platform. |
| Outcome: | The proposed system automates 45% of sessions and reduces average handling time by 39% without degrading support quality level. |
Multi-Agent Comedy Club: Investigating Community Discussion Effects on LLM Humor Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on the use of multi-turn interaction and feedback for LLM writing focus on prompts and localized feedback. |
| Approach: | They build a controlled multi-agent sandbox that instantiates a small standup comedy community and allows it to manipu-late whether public reception is generated, logged, and fed back into later rounds. |
| Outcome: | The proposed model improves craft/clarity and social response with occasional increases in aggressive humor. |
Critic-Driven Decoding for Mitigating Hallucinations in Data-to-text Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Hallucination of text lacking grounding in input data is a problem in neural data-to-text generation. |
| Approach: | They propose to combine probabilistic output of a generator language model with the output of an “text critic” classifier which guides the generation by assessing the match between the input data and the generated text. |
| Outcome: | The proposed method improves on the WebNLG and OpenDialKG benchmarks. |
Unsupervised Grammatical Error Correction Rivaling Supervised Methods (2023.emnlp-main)
Copied to clipboard
| Challenge: | Current state-of-the-art grammatical error correction systems rely on labeled data . current systems require manual correction and require a large quantity of labeles . |
| Approach: | They propose an unsupervised method to build a grammatical error correction system using a fixer and a critic. |
| Outcome: | The proposed system outperforms previous unsupervised systems on English and Chinese GEC. |
Dream to Chat: Model-based Reinforcement Learning on Dialogues with User Belief Modeling (2025.findings-emnlp)
Copied to clipboard
Yue Zhao, Xiaoyu Wang, Dan Wang, Zhonglin Jiang, Qingqing Gu, Teng Chen, Ningyuan Xi, Jinxian Qu, Yong Chen, Luo Ji
| Challenge: | a framework for constructing dialogue world models for natural language tasks is currently lacking. |
| Approach: | They propose a framework that can be used to train a dialogue world model. |
| Outcome: | The proposed framework can predict future utterances and user beliefs . it can achieve state-of-the-art performance on emotion classification and sentiment identification . |
Critic-Guided Decoding for Controlled Text Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work has demonstrated reinforcement learning and weighted decoding as effective approaches to achieve a higher level of language control and quality with pros and cons. |
| Approach: | They propose a method that combines reinforcement learning and weighted decoding to train a critic from reward models. |
| Outcome: | The proposed method generates more coherent and well-controlled texts than previous methods on three controlled generation tasks, topic control, sentiment control, and detoxification. |
SCRIPT: Self-Critic PreTraining of Transformers (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for representation learning of text are masked language modeling (MLM) a language model is trained to learn universal contextual embeddings, which are fine-tuned on a down-stream task. |
| Approach: | They propose a self-critic pretraining transformer for representation learning of text . they demonstrate improved sample-efficiency and improved performance over strong baselines . |
| Outcome: | The proposed model improves sample-efficiency and performance over strong baselines. |
No More Stale Feedback: Co-Evolving Critics for Open-World Agent Learning (2026.acl-long)
Copied to clipboard
Zhicong Li, Lingjie Jiang, Yulan Hu, Xingchen Zeng, Yixia Li, Xiangwen Zhang, Guanhua Chen, Zheng Pan, Xin Li, Yong Liu
| Challenge: | Current methods for training Large Language Model agents rely on static or offline critic models, which fail to adapt as the policy evolves. |
| Approach: | They propose a framework that integrates a critique and a policy to optimize the policy and critic through a synchronized co-evolutionary loop. |
| Outcome: | The proposed framework yields more stable training and higher long-horizon task success across open-world environments. |
Android Coach: Improve Online Agentic Training Efficiency with Single State Multiple Actions (2026.acl-long)
Copied to clipboard
| Challenge: | Existing reinforcement learning methods are expensive due to high latency and sample inefficiency . Currently, RL is limited to one-to-one state-action pairs . |
| Approach: | They propose a framework that shifts the training paradigm to Single State Multiple Actions and introduce a group-wise advantage estimator based on the averaged critic outputs. |
| Outcome: | The proposed framework achieves 7.5% and 8.3% success rate improvements on AndroidLab and AndroidWorld over UI-TARS-1.5-7B and attains 1.4x higher training efficiency than existing methods. |
CritiqueLLM: Towards an Informative Critique Generation Model for Evaluation of Large Language Model Generation (2024.acl-long)
Copied to clipboard
Pei Ke, Bosi Wen, Andrew Feng, Xiao Liu, Xuanyu Lei, Jiale Cheng, Shengyuan Wang, Aohan Zeng, Yuxiao Dong, Hongning Wang, Jie Tang, Minlie Huang
| Challenge: | Existing models for NLP evaluations lack the ability to generate informative critiques in pointwise grading and pairwise comparison especially without references. |
| Approach: | They propose a method which can acquire pointwise grading critiques with pseudo references and revise these critiques via multi-path prompting to obtain informative evaluation data in different tasks and settings. |
| Outcome: | The proposed method outperforms all open-source models and even GPT-4 in system-level correlations of pointwise grading. |
When LLMs Read Tables Carelessly: Measuring and Reducing Data Referencing Errors (2026.acl-long)
Copied to clipboard
Yuqing Yang, Qi Zhu, Zhen Han, Boran Han, Zhengyuan Shen, Shuai Wang, Vassilis N. Ioannidis, Huzefa Rangwala
| Challenge: | Large language models (LLMs) perform well on table tasks, but they still make data referencing errors (DREs) prior studies have only offered limited, small-scale analyses. |
| Approach: | They propose inference-time strategies and lightweight critics to mitigate data referencing errors. |
| Outcome: | The proposed model achieves an average F1 score of 78.2% in detecting both in-distribution and out-of-difference DREs and assists inference for larger models. |
Measure Twice, Click Once: Co-evolving Proposer and Visual Critic via Reinforcement Learning for GUI Grounding (2026.acl-long)
Copied to clipboard
| Challenge: | Graphical User Interface (GUI) grounding requires mapping natural language instructions to precise pixel coordinates due to visually homogeneous elements and dense layouts. |
| Approach: | They propose to replace static consistency strategies with a learnable selection mechanism that selects the optimal target by critiquing its own proposals rendered on the screenshot. |
| Outcome: | The proposed model significantly improves both grounding and critiquing capabilities over 6 benchmarks. |
RePair: Automated Program Repair with Process-based Feedback (2024.findings-acl)
Copied to clipboard
| Challenge: | Commercial-scale language models (LMs) have taken APR to unprecedented levels, but they are limited by parameters and humans interact with them through explicit prompts. |
| Approach: | They propose a method that utilizes process supervision to improve program repair by allowing users to input feedback from compilers and test cases. |
| Outcome: | The proposed method outperforms large outcome-based generation methods and is inspired by strategies used in programming competitions. |
ReLook: Vision-Grounded RL with a Multimodal LLM Critic for Agentic Web Coding (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) excel at algorithmic code generation, but front-end development is lacking in visual fidelity and interaction. |
| Approach: | They propose an agentic, vision-grounded reinforcement learning framework that closes a loop by invoking a multimodal LLM as a tool. |
| Outcome: | The proposed framework outperforms baselines in front-end code generation. |
VecCISC: Improving Confidence-Informed Self-Consistency with Reasoning Trace Clustering and Candidate Answer Selection (2026.findings-acl)
Copied to clipboard
| Challenge: | Weighted majority voting requires a critic to evaluate each candidate’s reasoning trace to produce the answer’s confidence score. |
| Approach: | They propose a lightweight framework that uses a measure of semantic similarity to filter reasoning traces that are semantically equivalent to others, degenerate, or hallucinated. |
| Outcome: | The proposed framework reduces token usage by 47% while maintaining or exceeding the accuracy of CISC. |
Debate-of-Thoughts: Resolving Knowledge Conflicts in LLMs Through Internal Deliberation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for retrieval augmented generation are based on a simplistic binary choice of relying on external contexts or memory. |
| Approach: | They propose a framework that transforms conflict resolution into an active deliberation process by incorporating contradictions as opportunities for deeper reasoning. |
| Outcome: | Experiments show that DoT outperforms state-of-the-art methods while generating transparent debate transcripts that explain its decisions. |