Papers by Ryan Steed
Upstream Mitigation Is Not All You Need: Testing the Bias Transfer Hypothesis in Pre-Trained Language Models (2022.acl-long)
Copied to clipboard
| Challenge: | Large language models and other massively pre-trained "foundation" models can easily adapt to a wide variety of downstream tasks in a process called finetuning. |
| Approach: | They propose to use the bias transfer hypothesis to reduce social biases internalized by large language models during pre-training into harmful task-specific behavior after fine-tuning. |
| Outcome: | The bias transfer hypothesis is the theory that social biases internalized by large language models during pre-training transfer into harmful task-specific behavior after fine-tuning. |