Papers by Weiyi Lu
DynaMaR: Dynamic Prompt with Mask Token Representation (2022.emnlp-industry)
Copied to clipboard
Xiaodi Sun, Sunny Rajagopalan, Priyanka Nigam, Weiyi Lu, Yi Xu, Iman Keivanloo, Belinda Zeng, Trishul Chilimbi
| Challenge: | Recent research shows that large language models pretrained using unsupervised approaches can achieve significant performance improvement on many downstream tasks. |
| Approach: | They propose an unsupervised approach to fine-tuning large language models using unsupervised approaches to many downstream tasks. |
| Outcome: | The proposed approach improves on four e-commerce applications and can achieve an average improvement of 10% in few-shot settings and 3.7% in data-rich settings over the standard approach. |
Similar but not the Same: Word Sense Disambiguation Improves Event Detection via Neural Representation Matching (D18-1)
Copied to clipboard
| Challenge: | Event detection (ED) and word sense disambiguation (WSD) are similar tasks, but they require different neural representations. |
| Approach: | They propose a method to transfer the knowledge learned on WSD to ED by matching neural representations learned for the two tasks. |
| Outcome: | The proposed method can be applied to event detection and word sense disambiguation datasets. |
Asynchronous Convergence in Multi-Task Learning via Knowledge Distillation from Converged Tasks (2022.naacl-industry)
Copied to clipboard
Weiyi Lu, Sunny Rajagopalan, Priyanka Nigam, Jaspreet Singh, Xiaodi Sun, Yi Xu, Belinda Zeng, Trishul Chilimbi
| Challenge: | Multi-task learning (MTL) aims to solve multiple tasks by sharing a base representation among them. |
| Approach: | They propose an approach that allows for "asynchronous" convergence among the tasks where each task can converge on its own schedule. |
| Outcome: | The proposed method outperforms existing methods in two 5-task MTL setups. |