Papers by Pete Walsh
Continued Pretraining for Better Zero- and Few-Shot Promptability (2022.emnlp-main)
Copied to clipboard
Zhaofeng Wu, Robert L Logan IV, Pete Walsh, Akshita Bhagia, Dirk Groeneveld, Sameer Singh, Iz Beltagy
| Challenge: | Recent language model prompting methods can achieve high accuracy in zero- and few-shot settings while requiring few to no learned task-specific parameters. |
| Approach: | They propose to use a dedicated pretraining stage to improve promptability in zero-shot settings and few-shot tuning. |
| Outcome: | The proposed method improves promptability in zero- and few-shot settings, while the existing method yields subpar performance. |