Papers by Rafael Rafailov
Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback (2023.emnlp-main)
Copied to clipboard
Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, Christopher Manning
| Challenge: | Recent studies have shown that unsupervised pre-training produces large language models whose conditional probabilities are remarkably well-calibrated. |
| Approach: | They propose to use verbalized confidences to extract confidence from large language models with reinforcement learning from human feedback to improve their accuracy. |
| Outcome: | The proposed methods reduce the expected calibration error by 50% for RLHF-LMs such as ChatGPT, GPT-4, and Claude. |
Disentangling Length from Quality in Direct Preference Optimization (2024.findings-acl)
Copied to clipboard
| Challenge: | Reinforcement Learning from Human Feedback (RLHF) exploits biases in human preferences, such as verbosity, and is under-explored for Direct Alignment Algorithms such as DPO. |
| Approach: | They propose to exploit verbosity biases in RLHF by using direct preference optimization to fine-tune models. |
| Outcome: | The proposed approach achieves 20% improvement in win rates when controlling for length on summarization and dialogue datasets, despite the GPT4 judge’s verbosity bias. |
PERSONA: A Reproducible Testbed for Pluralistic Alignment (2025.coling-main)
Copied to clipboard
| Challenge: | Currently, preference optimization approaches fail to capture the plurality of user opinions . Currently used methods do not account for the pluralities of users and difference of opinion . |
| Approach: | They propose a reproducible test bed to evaluate pluralistic alignment of language models . they generate user profiles from census data and use a large-scale evaluation dataset . |
| Outcome: | The proposed model improves pluralistic alignment of language models with diverse user values . it generates a large-scale evaluation dataset with 317,200 feedback pairs . |
LitBench: A Benchmark and Dataset for Reliable Evaluation of Creative Writing (2026.eacl-long)
Copied to clipboard
| Challenge: | a single prompt can inspire countless valid stories, making objective verification impossible. |
| Approach: | They propose a large-scale benchmark for creative writing evaluation using a reddit corpus and a 2,480-pair test set. |
| Outcome: | The proposed model outperforms existing OTS judges and generative reward models in the evaluation of creative writing. |