AutoRubric: Rubric-Based Generative Rewards for Faithful Multimodal Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Multimodal large language models (MLLMs) have advanced from perception tasks to complex multi-step reasoning. |
| Approach: | They propose a framework that integrates reinforcement learning with verifiable rewards with process-level supervision through automatically collected rubric-based generative rewards. |
| Outcome: | The proposed framework achieves state-of-the-art performance on six multimodal reasoning benchmarks and significantly improves reasoning faithfulness in dedicated evaluations. |
Similar Papers
OpenRubrics: Towards Scalable Synthetic Rubric Generation for Reward Modeling and LLM Alignment (2026.acl-long)
Copied to clipboard
| Challenge: | Existing reward models rely on scalar or pairwise judgments that fail to capture multifaceted nature of human preferences. |
| Approach: | They propose a rubric-based reward model that uses a large collection of prompt, rubric pairs to generate a scalar score or preference label for each response. |
| Outcome: | The proposed model surpasses strong size-matched baselines by 8.4% across multiple benchmarks. |
Crossing the Reward Bridge: Expanding Reinforcement Learning with Verifiable Rewards Across Diverse Domains (2026.acl-long)
Copied to clipboard
| Challenge: | Reinforcement learning with verifiable rewards (RLVR) has been effective on structured tasks, but its reliance on simple, rule-based verifiers creates a bottleneck. |
| Approach: | They propose a framework that uses a generative verifier to provide soft, probabilistic rewards. |
| Outcome: | The proposed framework outperforms existing models up to 10x their size and can be scalable and effective. |
Verifying the Subjective: Structured Multilingual Rewards for Low-Resource Alignment (2026.findings-acl)
Copied to clipboard
| Challenge: | Structured Multilingual Reward Modeling Framework extends Reinforcement Learning with Verifiable Rewards (RLVR) to subjective and open-ended tasks. |
| Approach: | They propose a framework that extends Reinforcement Learning with Verifiable Rewards to subjective and open-ended tasks. |
| Outcome: | The proposed framework improves reasoning capability and response quality on 7 tasks across 50 low-resource languages. |
GRPO-CARE: Consistency-Aware Reinforcement Learning for Multimodal Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent reinforcement learning approaches have advanced reasoning in Large Language Models (LLMs), yet their adaptation to multimodal LLMs remains underexplored. |
| Approach: | They propose a reinforcement learning framework that eliminates KL penalties and rewards consistency . they propose GRPO-CARE, which outperforms standard GR PO, with a base reward for accuracy and an adaptive bonus for consistency. |
| Outcome: | The proposed framework outperforms standard GRPO on the most difficult evaluation level and reasoning consistency test benchmarks. |
R-Search: Empowering LLM Reasoning with Search via Multi-Reward Reinforcement Learning (2026.findings-acl)
Copied to clipboard
Qingfei Zhao, Ruobing Wang, Dingling Xu, Daren Zha, Ma Bowen, Zhichun Wang, Shijie Jia, Limin Liu, Xin Wang
| Challenge: | Large language models (LLMs) have demonstrated impressive capabilities in multi-step and long-chain reasoning, but extending their reasoning capabilities to encompass deep interactions with search remains a non-trivial challenge. |
| Approach: | They propose a framework for Reasoning–Search integration that integrates multi-reward signals to optimize the reasoning–search interaction trajectories. |
| Outcome: | Experiments on seven datasets show that R-Search significantly outperforms mainstream RAG baselines. |
R2-MultiOmnia: Leading Multilingual Multimodal Reasoning via Self-Training (2025.acl-long)
Copied to clipboard
| Challenge: | Recent studies have introduced eclectic strategies to enhance MLLMs’ reasoning capabilities, but they remain related to a single language. |
| Approach: | They propose a modular approach that instructs models to abstract key elements of the reasoning process and refine reasoning trajectories via self-correction. |
| Outcome: | The proposed approach improves multimodal reasoning, gets aligned performances among the languages approaching strong models and improves the model's performance. |
DARL: Encouraging Diverse Answers for General Reasoning without Verifiers (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent efforts such as RLPR have extended RLVR to general domains, enabling training on broader datasets and achieving improvements over RL PR. |
| Approach: | They propose a framework that encourages the generation of diverse answers within a controlled deviation range from the reference while preserving alignment with it. |
| Outcome: | Extensive experiments on 13 benchmarks show that DARL surpasses RLPR in both reasoning accuracy and output diversity. |
Omni-R1: Towards the Unified Generative Paradigm for Multimodal Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Early approaches focus on text-based reasoning, but they often follow a single task-specific reasoning pattern. |
| Approach: | They propose a generative multimodal reasoning paradigm that unifies diverse reasoning skills by generating intermediate images during the reasoning process. |
| Outcome: | The proposed model unifies diverse multimodal reasoning skills by generating intermediate images during the reasoning process. |
Act as you think: Reinforcing Consistent Reasoning in Medical Visual Question Answering (2026.acl-long)
Copied to clipboard
Songtao Jiang, Yuan Wang, Ruizhe Chen, Yan Zhang, Ruilin Luo, Bohan Lei, Yeying Jin, Sibo Song, ZhiBo Yang, Jimeng Sun, Jian Wu, Zuozhu Liu
| Challenge: | Recent advances have improved the accuracy of medical visual question answering (Med-VQA) however, the high stakes nature of the medical domain has precipitated a shift towards interpretability and transparency of reasoning processes. |
| Approach: | They propose a reinforcement learning from verifiable rewards framework that rewards internal consistency and logical coherence. |
| Outcome: | The proposed framework rewards internal consistency and logical coherence, and is highly versatile, the authors show. |
RiT: Rubrics-in-Thinking Reinforcement Learning for Improved Reasoning in Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Reasoning Models benefit from generating intermediate reasoning steps alongside final answers. |
| Approach: | They propose a framework to introduce thinking-rubric supervision into intermediate reasoning. |
| Outcome: | The proposed framework outperforms outcome-only RL baselines on reasoning-intensive and open-ended tasks. |