Papers by Leon Ackermann
On the Relationship between Skill Neurons and Robustness in Prompt Tuning (2024.lrec-main)
Copied to clipboard
| Challenge: | Prompt Tuning is a parameter-efficient finetuning method for pre-trained large language models (PLMs). |
| Approach: | They propose to use RoBERTa to fine tune pre-trained large language models by finetuning only a small set of parameters to adjust for downstream tasks. |
| Outcome: | The proposed method activates specific neurons in the transformer’s feed-forward networks that are highly predictive and selective for the given task. |