Papers by Yiqiao Zhong
How does Multi-Task Training Affect Transformer In-Context Capabilities? Investigations with Function Classes (2024.naacl-short)
Copied to clipboard
| Challenge: | Multi-task learning (MTL) for generalist models is a promising direction that offers transfer learning potential. |
| Approach: | They propose to combine multi-task learning (MTL) with in-context learning (ICL) to build models that can generalize to multiple tasks while being robust to out-of-distribution examples. |
| Outcome: | The proposed training strategies enable models to learn difficult tasks while mixing in prior tasks, denoted as mixed curriculum. |