Papers by Stephanie Schoch
Data Selection for Fine-tuning Large Language Models Using Transferred Shapley Values (2023.acl-srw)
Copied to clipboard
| Challenge: | Large language models (LMs) have been shown to be highly effective for identifying harmful training instances, but dataset size and model complexity constraints limit the ability to apply Shapley-based data valuation to fine-tuning large pre-trained language models. |
| Approach: | They propose an algorithm that aggregates Shapley values from subsets for valuation of entire training set and a value transfer method that leverages value information extracted from a simple classifier trained using representations from the target language model. |
| Outcome: | The proposed method outperforms existing methods on benchmark datasets and can filter fine-tuning data to increase language model performance compared to training with the full fine-uning dataset. |
In-Context Learning (and Unlearning) of Length Biases (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing work has demonstrated the ability of large language models to learn lexical and label biases in-context negatively impacts performance and robustness of models. |
| Approach: | They investigate the impact of length biases on in-context learning by analyzing model length information in-constext. |
| Outcome: | The proposed model learns length biases in the context window without parameter updates. |
The Good, the Bad, and the Debatable: A Survey on the Impacts of Data for In-Context Learning (2025.emnlp-main)
Copied to clipboard
| Challenge: | In-context learning (ICL) is an emergent capability of large language models that allows them to learn new tasks at inference time without updating parameter updates. |
| Approach: | They propose to examine the relationship between data and in-context learning by examining the qualities of demonstrations that are desirable when selecting demonstrations, the "bad" qualities of demonstrators that can negatively impact the model and the "debatable" qualities. |
| Outcome: | The proposed model can learn unseen tasks by seeing a number of examples in the context window without updating parameters. |