P-Check: Advancing Personalized Reward Model via Learning to Generate Dynamic Checklist (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to personalized reward modeling treat user context as static or implicit conditioning signal, failing to capture dynamic nature of human judgment. |
| Approach: | They propose a personalized reward modeling framework that synthesizes dynamic evaluation criteria for guiding the reward prediction. |
| Outcome: | The proposed framework improves reward accuracy and enhances downstream personalized generation. |
Similar Papers
Persona-judge: Personalized Alignment of Large Language Models via Token-level Self-judgment (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for aligning language models with human preferences rely on reward signals and additional annotated data, limiting their scalability and adaptability to diverse human values. |
| Approach: | They propose a discriminative paradigm that leverages the intrinsic preference judgment capabilities of the model to align language models with human preferences. |
| Outcome: | The proposed model is scalable and efficient, paving the way for more adaptive personalized alignment. |
APLOT: Robust Reward Modeling via Adaptive Preference Learning with Optimal Transport (2025.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that RLHF improves performance of Large Language Models . BT-based RMs struggle to distinguish between similar preference responses . |
| Approach: | They propose to enhance BT-based reward models by using an adaptive margin mechanism . they use semantic similarity and reward-predicted reward differences to adjust focus . |
| Outcome: | Experimental results show that the proposed method outperforms existing methods in both in-distribution and OOD settings. |
AdaJudge: Adaptive Multi-Perspective Judging for Reward Modeling (2026.acl-long)
Copied to clipboard
| Challenge: | Existing reward models rely on a static pooling strategy to condense sequences into scalar scores, which is ill-suited for fine-grained discrimination. |
| Approach: | They propose a framework that jointly adapts representation and aggregation to address these limitations by integrating a static inductive bias with a representational mismatch. |
| Outcome: | Experiments on RM-Bench and JudgeBench show that AdaJudge outperforms strong off-the-shelf reward models and traditional pooling baselines. |
Check Your Work: Structured Checklist Feedback for Improving Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models have been driven by verifiable feedback in deterministic domains like mathematics and code. |
| Approach: | They propose to decompose granular, prompt-specific checklists into a scalar reward and use them to transform them into skalar rewards. |
| Outcome: | The proposed approach yields an 11.8% win-rate improvement on AlpacaEval 2.0 using Qwen3-8B, outperforming holistic reward models and existing checklist baselines. |
Teach a Reward Model to Correct Itself: Reward Guided Adversarial Failure Discovery for Robust Reward Modeling (2026.acl-long)
Copied to clipboard
| Challenge: | Existing failure discovery methods rely on prior knowledge of preference attributes . Existing methods do not scale to new models or data. |
| Approach: | They propose a preference distribution agnostic procedure that uses the reward model itself to guide controlled decoding toward mis specified responses while preserving the underlying preference class. |
| Outcome: | The proposed procedure improves robustness without degrading reward quality across models. |
AutoChecklist: Composable Pipelines for Checklist Generation and Scoring with LLM-as-a-Judge (2026.acl-demo)
Copied to clipboard
| Challenge: | AutoChecklist is an open-source library that unifies checklist-based evaluation into composable pipelines. |
| Approach: | They propose an open-source library that unifies checklist-based evaluation into composable pipelines. |
| Outcome: | The open-source library unifies checklist-based evaluation into composable pipelines. |
Act-Adaptive Margin: Dynamically Calibrating Reward Models for Subjective Ambiguity (2026.acl-long)
Copied to clipboard
Feiteng Fang, Dingwei Chen, Xiang Huang, Ting-En Lin, Yuchuan Wu, Xiong Liu, Jing Ye, Ziqiang Liu, Haonan Zhang, Liang Zhu, Hamid Alinejad-Rokny, Min Yang, Yongbin Li
| Challenge: | Existing approaches to reward modeling in reinforcement learning tasks are limited when dealing with ambiguous preferences. |
| Approach: | They propose to use AAM to dynamically calibrate preference margins using the Bradley-Terry model's internal parameter knowledge to improve reward modeling in subjective tasks. |
| Outcome: | The proposed approach improves reward modeling by dynamically calibrating preference margins using the model’s internal parameter knowledge. |
SynthesizeMe! Inducing Persona-Guided Prompts for Personalized Reward Models in LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Recent calls for pluralistic alignment of Large Language Models encourage adapting models to diverse user preferences. |
| Approach: | They propose a method to induce synthetic user personas from user interactions for personalized reward modeling. |
| Outcome: | The proposed approach improves LLM-as-a-judge accuracy by 4.4% on Chatbot Arena. |
PaTaRM: Bridging Pairwise and Pointwise Signals via Preference-Aware Task-Adaptive Reward Modeling (2026.acl-long)
Copied to clipboard
| Challenge: | Existing reward models lack generative and reasoning capabilities, resulting in poor performance. |
| Approach: | They propose a reward-aware task-adaptive reward model that enables pointwise training using readily available pairwise data via a novel Preference-Aware Reward mechanism. |
| Outcome: | The proposed reward model achieves an average relative improvement of 8.7% over the base models on RewardBench and RMBench. |
MiCRo: Mixture Modeling and Context-aware Routing for Personalized Preference Learning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing reward models assume a global reward function, limiting personalization and pluralistic alignment. |
| Approach: | They propose a framework that leverages binary preference datasets to enhance personalized preference learning. |
| Outcome: | The proposed framework captures diverse human preferences without fine-grained annotations and significantly improves personalized preference learning on downstream tasks. |