| Challenge: | Existing work on "pairwise" MTL has been validated in sequence tagging but key issues remain about its effectiveness. |
| Approach: | They propose three general multi-task learning approaches on 11 sequence tagging tasks. |
| Outcome: | The proposed approaches improve on 11 sequence tagging tasks. |
Similar Papers
A Survey of Multi-task Learning in Natural Language Processing: Regarding Task Relatedness and Training Methods (2023.eacl-main)
Copied to clipboard
| Challenge: | Multi-task learning is a popular approach in natural language processing because of its commonalities and differences. |
| Approach: | They propose to summarize recent advances in multi-task learning methods based on their task relatedness into two general multi-step training methods. |
| Outcome: | The proposed methods summarize the tasks and discuss future directions. |
Estimating the influence of auxiliary tasks for multi-task learning of sequence tagging tasks (2020.acl-main)
Copied to clipboard
| Challenge: | Multitask learning and transfer learning are techniques to overcome data scarcity . finding suitable auxiliary datasets for multitask learning is a trial-and-error approach . |
| Approach: | They propose to automatically assess the similarity of sequence tagging datasets to identify beneficial auxiliary data for MTL or TL setups. |
| Outcome: | The proposed methods can compute similarity between two sequence tagging datasets . they show that the same measures correlate with the change in test score of the auxiliary dataset . |
What can we learn from Semantic Tagging? (D18-1)
Copied to clipboard
| Challenge: | a recent study shows that multi-task learning improves performance of NLP tasks by exploiting similarities between tasks. |
| Approach: | They employ semantic tagging as an auxiliary task for three NLP tasks . they compare full neural network sharing, partial neural network shared and learning what to share . |
| Outcome: | The proposed model improves for part-of-speech tagging, universal dependency parsing and natural language inference. |
Multi-Task Learning for Argumentation Mining in Low-Resource Settings (N18-2)
Copied to clipboard
| Challenge: | Argument component identification is difficult for trained annotators to perform in a new domain or to develop new AM tasks. |
| Approach: | They investigate whether multi-task learning can improve performance on AM problems . they found that MTL performs particularly well when little training data is available for the main task . |
| Outcome: | The proposed approach performs better when little training data is available for the main task, a common scenario in AM. |
Sequence Labeling Parsing by Learning across Representations (P19-1)
Copied to clipboard
| Challenge: | Constituency and dependency parsing are the main abstractions for representing syntactic structure of sentences . constituency parsers are considered disjointed tasks, and their improvements have been obtained separately. |
| Approach: | They propose to add auxiliary loss to constituency parsing paradigms and explore a model that parses both paradigms at no cost. |
| Outcome: | The proposed model outperforms single-task models by 1.05 F1 points and 0.62 UAS points for constituency parsing and dependency parsers. |
When Does Aggregating Multiple Skills with Multi-Task Learning Work? A Case Study in Financial NLP (2023.acl-long)
Copied to clipboard
| Challenge: | Multi-task learning (MTL) is a machine learning paradigm where multiple learning tasks are optimized simultaneously, exploiting commonalities and differences across them. |
| Approach: | They propose a parameter-efficient MTL architecture to improve task aggregation and to include loosely related skills from multiple datasets. |
| Outcome: | The proposed architecture outperforms single-task learning (STL) and is expected to outperformed it. |
SC-LSTM: Learning Task-Specific Representations in Multi-Task Learning for Sequence Labeling (N19-1)
Copied to clipboard
| Challenge: | Multi-task learning (MTL) has been studied for sequence labeling tasks . auxiliary tasks are selected specifically to improve performance of a target task . |
| Approach: | They propose a shared-cell long-short-term memory cell which contains shared parameters that can learn from all tasks and task-specific parameters that could learn task-related information. |
| Outcome: | The proposed model can learn from all tasks and task-specific parameters. |
Bag-of-Words Transfer: Non-Contextual Techniques for Multi-Task Learning (D19-61)
Copied to clipboard
| Challenge: | Existing approaches to multi-task learning take advantage of transfer among tasks . generative reconstruction of the observations is not included in the standard framework . |
| Approach: | They propose to use a syntactically-oblivious pooling encoder and pre-trained word embeddings to improve sentence-level representations. |
| Outcome: | The proposed techniques yield similar performance on a universe of task combinations while reducing training time and model size. |
A Learnable Skill Combination Strategy for Multi-task Learning in Natural Language Understanding (2026.findings-acl)
Copied to clipboard
| Challenge: | a novel multi-task learning framework for domain-specific natural language understanding tasks addresses these limitations by combing multiple tasks into a single framework. |
| Approach: | They propose a multi-task learning framework that decomposes the language model into modular skill components and employs a dynamic, learnable skill-combination mechanism to adaptively handle diverse tasks. |
| Outcome: | The proposed framework surpasses conventional multi-task learning approaches in performance. |
Multiple Tasks Integration: Tagging, Syntactic and Semantic Parsing as a Single Task (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing systems that bypass intermediate levels of analysis are prone to error propagation and are therefore free from interference. |
| Approach: | They propose a multitask paradigm orthogonal to weight sharing that uses multiple tasks to process input iteratively but concurrently at multiple levels of analysis. |
| Outcome: | The proposed model uses reinforcement learning and release from sequential constraints to improve the quality of the syntactic and semantic parses. |