Papers with R**eward
ETR: Entropy Trend Reward for Efficient Chain-of-Thought Reasoning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods to shorten CoTs use length penalties or global entropy reduction . Existing approaches to CoT reasoning have significant practical drawbacks . |
| Approach: | They propose a method that shortens CoTs by length penalties or global entropy reduction . they integrate ETR into Group Relative Policy Optimization and evaluate it . |
| Outcome: | The proposed objective improves accuracy–efficiency trade-off by +9.9% while reducing CoT length by 67% across four benchmarks. |
DogeRM: Equipping Reward Models with Domain Knowledge through Model Merging (2024.emnlp-main)
Copied to clipboard
| Challenge: | Modern large language models (LLMs) showcase impressive capabilities across various tasks with aligning their behavior with human preferences. |
| Approach: | They propose a framework that integrates domain-specific knowledge into a general reward model by model merging. |
| Outcome: | The proposed framework improves performance across different benchmarks and provides detailed analysis showing the effects of model merging. |
MAESTRO: Meta-learning Adaptive Estimation of Scalarization Trade-offs for Reward Optimization (2026.acl-long)
Copied to clipboard
Yang Zhao, Hepeng Wang, Xiao Ding, Yangou Ouyang, Bibo Cai, Kai Xiong, Jinglong Gao, Zhouhao Sun, Li Du, Bing Qin, Ting Liu
| Challenge: | Group-Relative Policy Optimization (GRPO) has emerged as an efficient paradigm for aligning Large Language Models (LLMs), but its efficacy is confined to domains with verifiable ground truths. |
| Approach: | They propose a meta-cognitive orchestration layer that treats reward scalarization as a dynamic latent policy, leveraging the model’s terminal hidden states as 'a semantic bottleneck' . Across seven benchmarks, MAESTRO consistently outperforms single-reward and static multi-objective baselines while preserving the efficiency advantages of GRPO. |
| Outcome: | The proposed model outperforms single-reward and static multi-objective baselines while preserving efficiency advantages. |