Patient Outcome and Zero-shot Diagnosis Prediction with Hypernetwork-guided Multitask Learning (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent advances apply artificial intelligence to predict clinical events or infer the probable diagnosis for clinical decision support. |
| Approach: | They propose a hypernetwork-based approach that generates task-conditioned parameters and coefficients of multitask prediction heads to learn task-specific prediction and balance the multitask learning. |
| Outcome: | Experiments on clinical notes from the real-world MIMIC database show that the proposed model can achieve better performance than baselines and improve zero-shot prediction on unseen diagnoses. |
Similar Papers
Multi Task Learning For Zero Shot Performance Prediction of Multilingual Models (2022.acl-long)
Copied to clipboard
| Challenge: | Massively Multilingual Transformer based Language Models have been shown to be effective on zero-shot transfer across languages, though performance varies from language to language depending on pivot language(s) used for fine-tuning. |
| Approach: | They propose to combine multi-task learning problems with multi-lingual Transformers to model zero-shot transfer across languages. |
| Outcome: | The proposed model can predict zero-shot transfer across languages with a multi-task learning problem with pretraining data in very few languages. |
Hyper-X: A Unified Hypernetwork for Multi-Task Multilingual Transfer (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing multilingual models cannot fully leverage training data when it is available in different task-language combinations. |
| Approach: | They propose a single hypernetwork that unifies multi-task and multilingual learning with efficient adaptation. |
| Outcome: | The proposed model achieves the best or competitive gain when a mixture of multiple resources is available while being significantly more efficient than existing models. |
Neural Multi-Task Learning for Stance Prediction (D19-66)
Copied to clipboard
| Challenge: | Existing models for fact checking are limited in size due to limited data available . stance detection is a key component of fact checking for journalists and news agencies . |
| Approach: | They propose to use textual information from existing datasets to improve stance prediction. |
| Outcome: | The proposed model outperforms state-of-the-art systems on a public benchmark dataset by 6.0 and 14.4 points in weighting. |
Multi-Task Retrieval for Knowledge-Intensive Tasks (2021.acl-long)
Copied to clipboard
Jean Maillard, Vladimir Karpukhin, Fabio Petroni, Wen-tau Yih, Barlas Oguz, Veselin Stoyanov, Gargi Ghosh
| Challenge: | Knowledge-intensive tasks require large amounts of knowledge about the world . recent neural retrieval models achieve better results by learning directly from task-specific training data. |
| Approach: | They propose a multi-task trained neural retrieval model that can be universally trained on a wide variety of problems. |
| Outcome: | The proposed model outperforms specialised retrievers on a few-shot setting and matches or improves state-of-the-art on multiple benchmarks. |
Constrained Multi-Task Learning for Bridging Resolution (2022.acl-long)
Copied to clipboard
| Challenge: | bridging resolution is the task of recognizing and resolving bridling anaphors in a text. |
| Approach: | They propose a constrained multi-task learning framework for bridging resolution that exploits cross-task consistency constraints to guide the learning process and pre-train the entity coreference model on publicly available coreference data. |
| Outcome: | The proposed model achieves state-of-the-art on three standard evaluation corpora. |
Explaining the Effectiveness of Multi-Task Learning for Efficient Knowledge Extraction from Spine MRI Reports (2022.naacl-industry)
Copied to clipboard
Arijit Sehanobish, McCullen Sandora, Nabila Abraham, Jayashri Pawar, Danielle Torres, Anasuya Das, Murray Becker, Richard Herzog, Benjamin Odry, Ron Vianu
| Challenge: | Pretrained Transformer based models finetuned on domain specific corpora have changed the landscape of NLP but training or fine-tuning these models for individual tasks can be time consuming and resource intensive. |
| Approach: | They propose to use pretrained Transformer based models finetuned on domain specific corpora to train models for individual tasks. |
| Outcome: | The proposed model can match the performance of a task specific model when the task specific models show similar representations across all of their hidden layers and their gradients are aligned, i.e. their gradient follows the same direction. |
Joint Multitask Learning for Community Question Answering Using Task-Specific Embeddings (D18-1)
Copied to clipboard
| Challenge: | Stack-Overflow, Quora, and Yahoo! Answers forums are not moderated, which results in noisy and redundant content. |
| Approach: | They use deep neural networks to learn meaningful task-specific embeddings . they incorporate the embeddables into a conditional random field model . |
| Outcome: | The proposed task improves significantly across evaluation metrics. |
Gated Multi-Task Network for Text Classification (N18-2)
Copied to clipboard
| Challenge: | Existing approaches to multitask learning share the features without distinguishing the usefulness of the features, generating undesired interference between tasks. |
| Approach: | They propose to introduce a gate mechanism into multi-task CNN and propose a new gated sharing unit which can filter the feature flows between tasks and greatly reduce the interference. |
| Outcome: | The proposed approach can learn selection rules automatically and gain a great improvement over strong baselines. |
Towards Better Multi-task Learning: A Framework for Optimizing Dataset Combinations in Large Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Using a neural network, large language models can be trained on multiple tasks, allowing them to perform tasks efficiently. |
| Approach: | They propose a framework that leverages a neural network to select the best dataset combinations for enhancing multi-task learning (MTL) They propose to iteratively refine the selection, greatly improving efficiency while being model-, dataset-, and domain-independent. |
| Outcome: | The proposed framework iteratively refines the selection, greatly improving efficiency, while being model-, dataset-, and domain-independent. |
Tied Multitask Learning for Neural Speech Translation (N18-1)
Copied to clipboard
| Challenge: | Recent efforts in endangered language documentation focus on collecting spoken language resources . BULB project uses mobile app to collect spoken resources accompanied by spoken translations . |
| Approach: | They propose a model where the second task decoder receives information from the first task . they apply regularization that encourages transitivity and invertibility to the model . |
| Outcome: | The proposed model improves performance on low-resource speech transcription and translation tasks. |