Papers by Louis Castricato
trlX: A Framework for Large Scale Reinforcement Learning from Human Feedback (2023.emnlp-main)
Copied to clipboard
Alexander Havrilla, Maksym Zhuravinskyi, Duy Phung, Aman Tiwari, Jonathan Tow, Stella Biderman, Quentin Anthony, Louis Castricato
| Challenge: | Current RLHF paradigms rely on Proximal Policy Optimization (PPO), which quickly becomes a challenge to implement and scale up to large architectures. |
| Approach: | They propose an open-source framework for reinforcement learning from human feedback . it allows for offline fine-tuning of large language models . |
| Outcome: | The framework can be used to fine-tune models up to and exceeding 70 billion parameters. |
PERSONA: A Reproducible Testbed for Pluralistic Alignment (2025.coling-main)
Copied to clipboard
| Challenge: | Currently, preference optimization approaches fail to capture the plurality of user opinions . Currently used methods do not account for the pluralities of users and difference of opinion . |
| Approach: | They propose a reproducible test bed to evaluate pluralistic alignment of language models . they generate user profiles from census data and use a large-scale evaluation dataset . |
| Outcome: | The proposed model improves pluralistic alignment of language models with diverse user values . it generates a large-scale evaluation dataset with 317,200 feedback pairs . |