Challenge: prevailing RLHF methods such as PPO and DPO depend on large-scale binary preference annotations.
Approach: They propose a method which converts natural feedback into continuous preference trajectories and optimizes them using the novel TraceBias algorithm.
Outcome: The proposed approach outperforms PPO and DPO in a variety of domains and improves alignment by up to 7.6% across diverse LLMs and preference domains.

Similar Papers

T-REG: Preference Optimization with Token-Level Reward Regularization (2025.acl-long)

Copied to clipboard

Challenge: Reinforcement learning from human feedback (RLHF) is a dominant approach for large language models to follow instructions and produce meaningful alignment.
Approach: They propose a method that leverages human feedback to optimize large language models . they propose to use sequence-level and token-level rewards to optimize preference .
Outcome: The proposed method outperforms baseline methods on Alpaca Eval 2 and Arena-Hard benchmarks.
TLCR: Token-Level Continuous Reward for Fine-grained Reinforcement Learning from Human Feedback (2024.findings-acl)

Copied to clipboard

Challenge: Existing approaches to provide token-level rewards fail to account for varying degrees of preference inherent to each token.
Approach: They propose a reward model that uses a discriminator to assign token-based continuous rewards to each token considering the context.
Outcome: Extensive experiments show that the proposed reward model improves on open-ended language generation benchmarks.
Reward Generalization in RLHF: A Topological Perspective (2025.findings-acl)

Copied to clipboard

Challenge: Existing alignment methods share a common topology of information flow, but their alternatives have not been thoroughly explored.
Approach: They propose a theory of reward generalization in reinforcement learning from human feedback . they propose induced Bayesian networks to model the impact of dataset topologies on reward generalisation .
Outcome: The proposed method achieves an average win rate of 65% on three NLP tasks.
RLHF Algorithms Ranked: An Extensive Evaluation Across Diverse Tasks, Rewards, and Hyperparameters (2025.emnlp-industry)

Copied to clipboard

Challenge: Proximal Policy Optimization (PPO) has fallen out of favor for Large Language Models (LLMs), but its complexity and inefficiency have spurred the investigation of simpler alternatives.
Approach: They evaluate 17 RLHF algorithms on two benchmarks, OpenAI’s TL;DR Summarization and Anthropic’s Helpfulness / Harmlessness.
Outcome: The proposed methods are based on OpenAI’s TL;DR Summarization and Anthropic’s Helpfulness / Harmlessness benchmarks with two different reward models and a Rules based reward model.
PaTaRM: Bridging Pairwise and Pointwise Signals via Preference-Aware Task-Adaptive Reward Modeling (2026.acl-long)

Copied to clipboard

Challenge: Existing reward models lack generative and reasoning capabilities, resulting in poor performance.
Approach: They propose a reward-aware task-adaptive reward model that enables pointwise training using readily available pairwise data via a novel Preference-Aware Reward mechanism.
Outcome: The proposed reward model achieves an average relative improvement of 8.7% over the base models on RewardBench and RMBench.
Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards (2024.acl-long)

Copied to clipboard

Challenge: Reinforcement Learning from Human Feedback (RLHF) relies on scalar rewards to capture user preferences.
Approach: They propose a framework that integrates multi-objective reward modeling to represent diverse preference profiles.
Outcome: The proposed method improves performance across reward objectives and targets.
Prototypical Reward Network for Data-Efficient Model Alignment (2024.acl-long)

Copied to clipboard

Challenge: Reinforcement Learning from Human Feedback (RLHF) is a reward model that fine-tunes Large Language Models (LLMs) by utilizing Prototypical Networks.
Approach: They propose a framework utilizing Prototypical Networks to enhance reward models under limited human feedback, enabling more stable and reliable structural learning from fewer samples.
Outcome: The proposed framework improves reward models under limited human feedback, surpassing traditional methods, especially in data-limited scenarios.
Self-supervised Preference Optimization: Enhance Your Language Model with Preference Degree Awareness (2024.findings-emnlp)

Copied to clipboard

Challenge: Recent studies have focused on replacing the reward model in Reinforcement Learning with Human Feedback (RLHF) methods for Large Language Models (LLMs).
Approach: They propose a self-supervised preference optimization framework that replaces the reward model with a preference loss and alignment loss to improve LLMs' ability to understand human preferences.
Outcome: The proposed framework can be integrated with existing preference optimization methods and significantly boost their performance.
SteerLM: Attribute Conditioned SFT as an (User-Steerable) Alternative to RLHF (2023.findings-emnlp)

Copied to clipboard

Challenge: supervised fine-tuning and reinforcement learning from human feedback (RLHF) are not effective in generating useful and high-quality responses.
Approach: They propose a supervised fine-tuning method that empowers end-users to control responses during inference.
Outcome: Experiments show that supervised fine-tuning and reinforcement learning from human feedback (RLHF) can generate helpful and high-quality responses while maintaining customizability.
APLOT: Robust Reward Modeling via Adaptive Preference Learning with Optimal Transport (2025.emnlp-main)

Copied to clipboard

Challenge: Experimental results show that RLHF improves performance of Large Language Models . BT-based RMs struggle to distinguish between similar preference responses .
Approach: They propose to enhance BT-based reward models by using an adaptive margin mechanism . they use semantic similarity and reward-predicted reward differences to adjust focus .
Outcome: Experimental results show that the proposed method outperforms existing methods in both in-distribution and OOD settings.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations