Papers by Simon Du
Reflect-RL: Two-Player Online RL Fine-Tuning for LMs (2024.acl-long)
Copied to clipboard
| Challenge: | supervised fine-tuning (SFT) on a limited offline dataset does not yield good performance. |
| Approach: | They propose a two-player system to fine-tune an LM using SFT and online RL . they use negative example generation to enhance error-correction ability of the reflection model . |
| Outcome: | The proposed system outperforms SFT and online RL without reflection on a GPT-2 XL 1.56B model. |
An Experimental Design Framework for Label-Efficient Supervised Finetuning of Large Language Models (2024.findings-acl)
Copied to clipboard
Gantavya Bhatt, Yifang Chen, Arnav Das, Jifan Zhang, Sang Truong, Stephen Mussmann, Yinglun Zhu, Jeff Bilmes, Simon Du, Kevin Jamieson, Jordan Ash, Robert Nowak
| Challenge: | Supervised finetuning (SFT) on instruction datasets has shown immense potential in improving the zero-shot generalization capabilities observed in large language models (LLMs). |
| Approach: | They propose to use experimental design to minimize the computational cost of active learning by identifying useful subsets of samples to annotate from an unlabeled pool. |
| Outcome: | The proposed methods save 50% of the annotation cost compared to random sampling on generative tasks. |