Papers by Leonard Lausen
Exploring the Role of Task Transferability in Large-Scale Multi-Task Learning (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent work has found that multi-task training with a large number of diverse tasks can uniformly improve downstream performance on unseen target tasks. |
| Approach: | They aim to disentangle the effect of scale and relatedness of tasks in multi-task representation learning by increasing the number of tasks and incorporating smaller sets of related tasks. |
| Outcome: | The proposed model improves on unseen target tasks by increasing the scale of multi-task learning to incorporate more tasks and developing similarity metrics to incorporate tasks related to the target task. |
Understanding Silent Data Corruption in LLM Training (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are a challenging task because of their large size and complexity. |
| Approach: | They propose to isolate and analyze the impact of SDCs on LLM training by using a cloud computing platform to access unhealthy nodes swept out of production by automated fleet management. |
| Outcome: | The proposed model training compares healthy production nodes with unhealthy nodes exhibiting SDCs at three levels: at each submodule computation, at a single optimizer step, and at . training period. |
Dive into Deep Learning for Natural Language Processing (D19-2)
Copied to clipboard
| Challenge: | GluonNLP is a powerful new toolkit that automates the most laborious aspects of deep learning for NLP. |
| Approach: | This hands-on tutorial demonstrates how to scale unsupervised pre-training techniques with Apache MXNet and GluonNLP. |
| Outcome: | This hands-on tutorial examines the challenges of scaling these models and algorithms effectively with Apache MXNet and GluonNLP. |