Papers by Albert Webson
PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts (2022.acl-demo)
Copied to clipboard
Stephen Bach, Victor Sanh, Zheng Xin Yong, Albert Webson, Colin Raffel, Nihal V. Nayak, Abheesht Sharma, Taewoon Kim, M Saiful Bari, Thibault Fevry, Zaid Alyafeai, Manan Dey, Andrea Santilli, Zhiqing Sun, Srulik Ben-david, Canwen Xu, Gunjan Chhablani, Han Wang, Jason Fries, Maged Al-shaibani, Shanya Sharma, Urmish Thakker, Khalid Almubarak, Xiangru Tang, Dragomir Radev, Mike Tian-jian Jiang, Alexander Rush
| Challenge: | PromptSource is a system for creating, sharing, and using natural language prompts . prompts are used to train and query language models in zero-shot learning settings . |
| Approach: | PromptSource is a system for creating, sharing, and using natural language prompts . et al.: using prompts to train and query language models is emerging area in NLP . they propose a templating language for defining data-linked prompts, a user interface that iterates on prompt development . |
| Outcome: | PromptSource is a system for creating, sharing, and using natural language prompts . it has a templating language for defining data-linked prompts and a community-driven set of guidelines . |
Are Language Models Worse than Humans at Following Prompts? It’s Complicated (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work shows that language models perform surprisingly well when given intentionally irrelevant or misleading prompts. |
| Approach: | They challenge an assumption that humans would perform badly when given pathological instructions by ignoring irrelevant prompts and following them faithfully when given misleading instructions. |
| Outcome: | The proposed model performs well when given intentionally irrelevant or misleading prompts, whereas models do not. |
Do Prompt-Based Models Really Understand the Meaning of Their Prompts? (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent studies show that prompts help models to learn faster in the same way that humans learn faster when provided with task instructions expressed in natural language. |
| Approach: | They experiment with 30 prompts manually written for natural language inference (NLI) they find that models can learn just as fast with many irrelevant or pathologically misleading prompts . |
| Outcome: | The proposed model can learn as fast with irrelevant or pathologically misleading prompts as with instructively “good” prompts. |
In-context Learning Generalizes, But Not Always Robustly: The Case of Syntax (2024.naacl-long)
Copied to clipboard
| Challenge: | In-context learning is a common method for teaching large language models new tasks . given labeled examples in the input context, the model learns to perform the task without weight updates. |
| Approach: | They examine whether models guided via ICL infer the underlying structure of the task defined by the context or rely on superficial heuristics that only generalize to identically distributed examples. |
| Outcome: | The proposed model generalizes syntactically or linearly on out-of-distribution examples . the proposed model is able to generalize better on pre-trained models . |
Crosslingual Generalization through Multitask Finetuning (2023.acl-long)
Copied to clipboard
Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng Xin Yong, Hailey Schoelkopf, Xiangru Tang, Dragomir Radev, Alham Fikri Aji, Khalid Almubarak, Samuel Albanie, Zaid Alyafeai, Albert Webson, Edward Raff, Colin Raffel
| Challenge: | Multitask prompted finetuning (MTF) has been shown to help large language models generalize to new tasks in a zero-shot setting, but so far explorations of MTF have focused on English data and models. |
| Approach: | They apply multitask prompted finetuning to pretrained multilingual models and generate variants called BLOOMZ and mT0. |
| Outcome: | The proposed models can generalize to non-English languages that have never been seen before. |
Are “Undocumented Workers” the Same as “Illegal Aliens”? Disentangling Denotation and Connotation in Vector Spaces (2020.emnlp-main)
Copied to clipboard
| Challenge: | popular pretrained models encode both denotation and connotation as one entangled representation . a researcher using a pretrained representation can confuse words with connotations . |
| Approach: | They propose a nerual netowrk that decomposes a pretrained representation as independent denotation and connotation representations. |
| Outcome: | The proposed model improves document rankings by comparing denotation and connotation representations with extrinsic representations. |