Papers by Alexander Havrilla
trlX: A Framework for Large Scale Reinforcement Learning from Human Feedback (2023.emnlp-main)
Copied to clipboard
Alexander Havrilla, Maksym Zhuravinskyi, Duy Phung, Aman Tiwari, Jonathan Tow, Stella Biderman, Quentin Anthony, Louis Castricato
| Challenge: | Current RLHF paradigms rely on Proximal Policy Optimization (PPO), which quickly becomes a challenge to implement and scale up to large architectures. |
| Approach: | They propose an open-source framework for reinforcement learning from human feedback . it allows for offline fine-tuning of large language models . |
| Outcome: | The framework can be used to fine-tune models up to and exceeding 70 billion parameters. |