| Challenge: | In multi-task learning, multiple related tasks are learned together. |
| Approach: | They propose methods that take advantage of natural groupings of related tasks . they propose parallel and serial architectures that can learn different feature spaces . |
| Outcome: | The proposed methods improve performance on natural language understanding (NLU) tasks. |
Similar Papers
Multi-Task Retrieval for Knowledge-Intensive Tasks (2021.acl-long)
Copied to clipboard
Jean Maillard, Vladimir Karpukhin, Fabio Petroni, Wen-tau Yih, Barlas Oguz, Veselin Stoyanov, Gargi Ghosh
| Challenge: | Knowledge-intensive tasks require large amounts of knowledge about the world . recent neural retrieval models achieve better results by learning directly from task-specific training data. |
| Approach: | They propose a multi-task trained neural retrieval model that can be universally trained on a wide variety of problems. |
| Outcome: | The proposed model outperforms specialised retrievers on a few-shot setting and matches or improves state-of-the-art on multiple benchmarks. |
What can we learn from Semantic Tagging? (D18-1)
Copied to clipboard
| Challenge: | a recent study shows that multi-task learning improves performance of NLP tasks by exploiting similarities between tasks. |
| Approach: | They employ semantic tagging as an auxiliary task for three NLP tasks . they compare full neural network sharing, partial neural network shared and learning what to share . |
| Outcome: | The proposed model improves for part-of-speech tagging, universal dependency parsing and natural language inference. |
Killing Four Birds with Two Stones: Multi-Task Learning for Non-Literal Language Detection (C18-1)
Copied to clipboard
| Challenge: | idioms and metaphors are often studied in isolation, challenging the distinction . e.g., metaphorical concept mappings are ubiquitous in everyday life, thus they are ubiquitous . |
| Approach: | They propose to view the detection problem as a generalized non-literal language classification problem. |
| Outcome: | The proposed model improves on four metaphor and idiom detection tasks in two languages, English and German. |
A Survey of Multi-task Learning in Natural Language Processing: Regarding Task Relatedness and Training Methods (2023.eacl-main)
Copied to clipboard
| Challenge: | Multi-task learning is a popular approach in natural language processing because of its commonalities and differences. |
| Approach: | They propose to summarize recent advances in multi-task learning methods based on their task relatedness into two general multi-step training methods. |
| Outcome: | The proposed methods summarize the tasks and discuss future directions. |
Towards Better Multi-task Learning: A Framework for Optimizing Dataset Combinations in Large Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Using a neural network, large language models can be trained on multiple tasks, allowing them to perform tasks efficiently. |
| Approach: | They propose a framework that leverages a neural network to select the best dataset combinations for enhancing multi-task learning (MTL) They propose to iteratively refine the selection, greatly improving efficiency while being model-, dataset-, and domain-independent. |
| Outcome: | The proposed framework iteratively refines the selection, greatly improving efficiency, while being model-, dataset-, and domain-independent. |
Tied Multitask Learning for Neural Speech Translation (N18-1)
Copied to clipboard
| Challenge: | Recent efforts in endangered language documentation focus on collecting spoken language resources . BULB project uses mobile app to collect spoken resources accompanied by spoken translations . |
| Approach: | They propose a model where the second task decoder receives information from the first task . they apply regularization that encourages transitivity and invertibility to the model . |
| Outcome: | The proposed model improves performance on low-resource speech transcription and translation tasks. |
Explaining the Effectiveness of Multi-Task Learning for Efficient Knowledge Extraction from Spine MRI Reports (2022.naacl-industry)
Copied to clipboard
Arijit Sehanobish, McCullen Sandora, Nabila Abraham, Jayashri Pawar, Danielle Torres, Anasuya Das, Murray Becker, Richard Herzog, Benjamin Odry, Ron Vianu
| Challenge: | Pretrained Transformer based models finetuned on domain specific corpora have changed the landscape of NLP but training or fine-tuning these models for individual tasks can be time consuming and resource intensive. |
| Approach: | They propose to use pretrained Transformer based models finetuned on domain specific corpora to train models for individual tasks. |
| Outcome: | The proposed model can match the performance of a task specific model when the task specific models show similar representations across all of their hidden layers and their gradients are aligned, i.e. their gradient follows the same direction. |
Multi-Task Learning for Sequence Tagging: An Empirical Study (C18-1)
Copied to clipboard
| Challenge: | Existing work on "pairwise" MTL has been validated in sequence tagging but key issues remain about its effectiveness. |
| Approach: | They propose three general multi-task learning approaches on 11 sequence tagging tasks. |
| Outcome: | The proposed approaches improve on 11 sequence tagging tasks. |
Learning What to Share: Leaky Multi-Task Network for Text Classification (C18-1)
Copied to clipboard
| Challenge: | Existing approaches to multi-task learning suffer from the interference between tasks because they lack selection mechanism for feature sharing. |
| Approach: | They propose a multi-task convolutional neural network with the Leaky Unit which has memory and forgetting mechanism to filter the feature flows between tasks. |
| Outcome: | The proposed model can filter feature flows between tasks and improve performance on five datasets. |
Cross-Task Generalization Abilities of Large Language Models (2024.naacl-srw)
Copied to clipboard
| Challenge: | a thesis proposal advocates for the crucial role of cross-task generalization in NLP systems. |
| Approach: | They propose to benchmark cross-task generalization abilities with diverse NLP tasks . they also propose to develop model architectures for improving cross- task generalization . |
| Outcome: | This paper compares cross-task generalization abilities with diverse NLP tasks . it also analyzes and predicts the generalization landscape of current state-of-the-art large language models . |