Papers by Stephen Chen
Rethinking Data Mixing from the Perspective of Large Language Models (2026.acl-short)
Copied to clipboard
Yuanjian Xu, Tianze Sun, Changwei Xu, XinLong Zhao, Jianing Hao, Ran Chen, Yang Liu, Ruijie Xu, Stephen Chen, Guang Zhang
| Challenge: | Existing methods to mix data with LLMs have relied on domain definitions derived from intuition. |
| Approach: | They propose a reweighting framework that restructures data scheduling as a graph-constrained optimization problem. |
| Outcome: | The proposed framework achieves competitive performance on GPT-2 models. |
What does the language of foods say about us? (D19-62)
Copied to clipboard
| Challenge: | Using a dataset of 24 million food-related tweets, we can predict if states in the United States are above the median rates for type 2 diabetes mellitus (T2DM) income, poverty, and education are important factors in predicting T2DM rates, but socioeconomic factors do not capture this information. |
| Approach: | They use a dataset of 24 million food-related tweets to investigate the signal contained in the language of food on social media. |
| Outcome: | The language of food can predict health risks, political orientation, and geographic location, and outperform previous work by 4–18%. |
An Experimental Design Framework for Label-Efficient Supervised Finetuning of Large Language Models (2024.findings-acl)
Copied to clipboard
Gantavya Bhatt, Yifang Chen, Arnav Das, Jifan Zhang, Sang Truong, Stephen Mussmann, Yinglun Zhu, Jeff Bilmes, Simon Du, Kevin Jamieson, Jordan Ash, Robert Nowak
| Challenge: | Supervised finetuning (SFT) on instruction datasets has shown immense potential in improving the zero-shot generalization capabilities observed in large language models (LLMs). |
| Approach: | They propose to use experimental design to minimize the computational cost of active learning by identifying useful subsets of samples to annotate from an unlabeled pool. |
| Outcome: | The proposed methods save 50% of the annotation cost compared to random sampling on generative tasks. |