Papers by Konstantin Usevich
Low-Rank Updates of pre-trained Weights for Multi-Task Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Multi-task learning is a popular approach for learning with pre-trained models due to the complexity of the tasks and the challenges associated with fine-tuning large pre-train models. |
| Approach: | They propose a new approach for Multi-task learning which is based on stacking the weights of Neural Networks as a tensor. |
| Outcome: | The proposed approach achieves equivalent performance to the state-of-the-art on the general language understanding evaluation benchmark by training only 0.3 of the parameters per task while not modifying the baseline weights. |