MTL

Generating Hypothetical Events for Abductive Inference (2021.starsem-1)

Challenge:	Argument component identification is difficult for trained annotators to perform in a new domain or to develop new AM tasks.
Approach:	They investigate whether multi-task learning can improve performance on AM problems . they found that MTL performs particularly well when little training data is available for the main task .
Outcome:	The proposed approach performs better when little training data is available for the main task, a common scenario in AM.

Copied to clipboard

Debjit Paul, Anette Frank

SEPSIS: I Can Catch Your Lies – A New Paradigm for Deception Detection (2025.acl-srw)

Challenge:	Abductive reasoning is inference to the best explanation given an incomplete set of observations about everyday situations.
Approach:	They propose a model that generates what could happen next from a hypothetical scenario and then proposes the most plausible explanation from varying hypothetical scenarios.
Outcome:	The proposed model improves over previous vanilla pre-trained models fine-tuned on Abductive NLI.

Copied to clipboard

Anku Rani, Dwip Dalal, Shreya Gautam, Pankaj Gupta, Vinija Jain, Aman Chadha, Amit Sheth, Amitava Das

Challenge:	a new framework categorizes deception into three forms: lies of omission, lies of commission, and lies of influence . a novel framework for deception detection leveraging NLP techniques is proposed .
Approach:	They propose a framework that categorizes deception into three forms: lies of omission, lies of commission, and lies of influence.
Outcome:	The proposed framework achieves an impressive F1 score of 0.87 across all layers . it can be used to investigate lies of omission, lies of commission and lies of influence .

A Probabilistic Toolkit for Multi-grained Word Segmentation in Chinese (2025.coling-demos)

Copied to clipboard

Xi Ma, Yang Hou, Xuebin Wang, Zhenghua Li

Syntactic Text Processing

Challenge:	Existing tools for word segmentation are based on different linguistic theories or target different scenarios.
Approach:	They propose a probabilistic toolkit for multi-grained word segmentation in Chinese . they adopt semi-Markov CRF for single-grain word segmenting (SWS) .
Outcome:	The proposed approach can produce marginal probabilities of words during inference and significantly improve performance in the cross-domain scenario.

In-BoXBART: Get Instructions into Biomedical Multi-Task Learning (2022.findings-naacl)

Copied to clipboard

Mihir Parmar, Swaroop Mishra, Mirali Purohit, Man Luo, Murad Mohammad, Chitta Baral

How does Multi-Task Training Affect Transformer In-Context Capabilities? Investigations with Function Classes (2024.naacl-short)

Challenge:	Experimental results show that the proposed model outperforms single-task baseline by 3% and multi-task (without instruction) baseline by 18% on an average.
Approach:	They propose a unified model that can learn all 32 instruction tasks of the BoX without any task-specific modules.
Outcome:	The proposed model outperforms single-task baseline by 3% and multi-task (without instruction) baseline by 18% on an average.

Copied to clipboard

Harmon Bhasin, Timothy Ossowski, Yiqiao Zhong, Junjie Hu

One Semantic Parser to Parse Them All: Sequence to Sequence Multi-Task Learning on Semantic Parsing Datasets (2021.starsem-1)

Challenge:	Multi-task learning (MTL) for generalist models is a promising direction that offers transfer learning potential.
Approach:	They propose to combine multi-task learning (MTL) with in-context learning (ICL) to build models that can generalize to multiple tasks while being robust to out-of-distribution examples.
Outcome:	The proposed training strategies enable models to learn difficult tasks while mixing in prior tasks, denoted as mixed curriculum.

Copied to clipboard

Marco Damonte, Emilio Monti

Adaptively Scheduled Multitask Learning: The Case of Low-Resource Neural Machine Translation (D19-56)

Challenge:	Existing semantic parsing datasets lack a single standard for meaning representations . lack of a standard led to the creation of plethora of datasets requiring expert annotators .
Approach:	They propose to use multi-task learning to unify different datasets and train a single model for them.
Outcome:	The proposed architectures yield better parsing accuracies and composition generalization than single-task models.

Copied to clipboard

Poorya Zaremoodi, Gholamreza Haffari

Multilinguality

Challenge:	Neural Machine Translation suffers from the lack of bilingual data in low-resource scenarios.
Approach:	They propose to inject inductive biases into Neural Machine Translation (NMT) using auxiliary syntactic and semantic tasks.
Outcome:	The proposed approach improves translation quality by reweighing training data of main and auxiliary tasks based on their contributions to generalisability of main task.

Asynchronous Convergence in Multi-Task Learning via Knowledge Distillation from Converged Tasks (2022.naacl-industry)

Copied to clipboard

Weiyi Lu, Sunny Rajagopalan, Priyanka Nigam, Jaspreet Singh, Xiaodi Sun, Yi Xu, Belinda Zeng, Trishul Chilimbi

A Match Made in Heaven: A Multi-task Framework for Hyperbole and Metaphor Detection (2023.findings-acl)

Challenge:	Multi-task learning (MTL) aims to solve multiple tasks by sharing a base representation among them.
Approach:	They propose an approach that allows for "asynchronous" convergence among the tasks where each task can converge on its own schedule.
Outcome:	The proposed method outperforms existing methods in two 5-task MTL setups.

Copied to clipboard

Naveen Badathala, Abisek Rajakumar Kalarani, Tejpalsingh Siledar, Pushpak Bhattacharyya

Geolocation with Attention-Based Multitask Learning Models (D19-55)

Challenge:	Existing approaches to detect metaphor and hyperbole independently have not explored their relationship computationally.
Approach:	They propose a multi-task deep learning framework to detect hyperbole and metaphor simultaneously by annotating two hyperbolic datasets with metaphor labels.
Outcome:	The proposed framework improves state-of-the-art hyperbole detection by 12% over existing methods.

Copied to clipboard

Tommaso Fornaciari, Dirk Hovy

Challenge:	predicting the location of a social media post requires discretization of the coordinates, but results in poor performance.
Approach:	They propose to combine two approaches to predict location using supervised models . they evaluate a multitask convolutional neural network that predicts both discrete locations and continuous coordinates .
Outcome:	The proposed model outperforms singletask models and prior work on one dataset and shows that correlation between labels and coordinates has a marked impact on the effectiveness of a regression task.

When to Use Multi-Task Learning vs Intermediate Fine-Tuning for Pre-Trained Encoder Transfer Learning (2022.acl-short)

Copied to clipboard

Orion Weller, Kevin Seppi, Matt Gardner

R-LoRA: Randomized Multi-Head LoRA for Efficient Multi-task Learning (2025.findings-emnlp)

Challenge:	Transfer learning (TL) in natural language processing has seen a surge of interest in recent years . pre-trained models have shown impressive ability to transfer to novel tasks .
Approach:	They compare two different methods of transfer learning in natural language processing to find out which is better.
Outcome:	The proposed methods perform better when the target task has fewer instances than the supporting task and vice versa.

Copied to clipboard

Jinda Liu, Yi Chang, Yuan Wu

English-to-Chinese Transliteration with Phonetic Auxiliary Task (2020.aacl-main)

Challenge:	Low-Rank Adaptation (LoRA) improves performance in multi-task learning by diversifying the head matrices through Multi-Head Dropout and Multi-head Random Initialization.
Approach:	They propose a low-rank adaptive approach to fine-tune large language models by approximating weight updates through low-ranked matrices.
Outcome:	The proposed approach improves performance in multi-task learning while reducing memory usage and training time.

Copied to clipboard

Yuan He, Shay B. Cohen

Syntactic Text Processing

Challenge:	Several NMT techniques have been used to enhance machine transliteration models, but few focus on the linguistic features specific to the relevant languages.
Approach:	They propose a phonetic auxiliary task that integrates phonetic features into a model to improve generalization performance of the main transliteration task.
Outcome:	The proposed model achieves similar performance to the current state of the art with a much smaller size.

An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models (2020.tacl-1)

Copied to clipboard

Lifu Tu, Garima Lalwani, Spandana Gella, He He

SRL4ORL: Improving Opinion Role Labeling Using Multi-Task Learning with Semantic Role Labeling (N18-1)

Challenge:	Recent work shows that pre-trained language models perform poorly on challenging datasets where spurious correlations do not hold.
Approach:	They propose to use multi-task learning to improve generalization from minority examples . they propose to combine MTL with auxiliary tasks to improve performance .
Outcome:	The proposed model generalizes from minority examples without hurting in-distribution performance.

Copied to clipboard

Ana Marasović, Anette Frank

A Survey of Multi-task Learning in Natural Language Processing: Regarding Task Relatedness and Training Methods (2023.eacl-main)

Challenge:	Recent neural approaches do not outperform the state-of-the-art feature-based models for Opinion Role Labeling (ORL).
Approach:	They propose to use multi-task learning to improve Opinion Role Labeling by using a related task which has substantially more data.
Outcome:	The proposed model outperforms the state-of-the-art model for Opinion Role Labeling (ORL) with more data.

Copied to clipboard

Zhihan Zhang, Wenhao Yu, Mengxia Yu, Zhichun Guo, Meng Jiang

XLPT-AMR: Cross-Lingual Pre-Training via Multi-Task Learning for Zero-Shot AMR Parsing and Text Generation (2021.acl-long)

Challenge:	Multi-task learning is a popular approach in natural language processing because of its commonalities and differences.
Approach:	They propose to summarize recent advances in multi-task learning methods based on their task relatedness into two general multi-step training methods.
Outcome:	The proposed methods summarize the tasks and discuss future directions.

Copied to clipboard

Dongqin Xu, Junhui Li, Muhua Zhu, Min Zhang, Guodong Zhou

Multi-task Learning for Multilingual Neural Machine Translation (2020.emnlp-main)

Challenge:	Abstract Meaning Representation (AMR) research is limited and challenging for languages other than English.
Approach:	They propose a cross-lingual pre-training approach for AMR parsing and text generation . they use an English-to-English parallel dataset and a multi-task learning approach .
Outcome:	The proposed approach outperforms baseline pre-training methods on English parsing and text generation tasks.

Copied to clipboard

Yiren Wang, ChengXiang Zhai, Hany Hassan

Multilinguality

Challenge:	Existing multilingual neural machine translation systems rely on bitext training data, which is limited and costly to collect.
Approach:	They propose a multi-task learning framework that trains the model with the translation task on bitext data and two denoising tasks on monolingual data.
Outcome:	The proposed framework outperforms pre-training models for both NMT and cross-lingual transfer learning NLU tasks.

MetaGPT: Merging Large Language Models Using Model Exclusive Task Arithmetic (2024.emnlp-main)

Copied to clipboard

Yuyan Zhou, Liang Song, Bingning Wang, Weipeng Chen

Adaptive Knowledge Sharing in Multi-Task Learning: Improving Low-Resource Neural Machine Translation (P18-2)

Challenge:	Existing methods face the trilemma of performance, data privacy, and computational costs, which hinders their application to LLMs.
Approach:	They propose a model-exclusive task arithmetic method for merging GPT-scale models which is data-agnostic and bypasses the heavy search process.
Outcome:	The proposed method achieves state-of-the-art performance on multiple tasks while minimizing the average loss difference between the merged model and each individual task model.

Copied to clipboard

Poorya Zaremoodi, Wray Buntine, Gholamreza Haffari

Many Hands Make Light Work: Using Essay Traits to Automatically Score Essays (2022.naacl-main)

Challenge:	Neural Machine Translation (NMT) requires large amounts of bilingual data to learn a translation model with reasonable quality.
Approach:	They propose to extend recurrent units with multiple "blocks" along with a trainable "routing network" this allows for adaptive collaboration by dynamic sharing of blocks conditioned on the task at hand, input, and model state.
Outcome:	Empirical evaluations of two low-resource translation tasks show +1 BLEU score improvements compared to strong baselines.

Copied to clipboard

Rahul Kumar, Sandeep Mathias, Sriparna Saha, Pushpak Bhattacharyya

Coupled Hierarchical Transformer for Stance-Aware Rumor Verification in Social Media Conversations (2020.emnlp-main)

Challenge:	In automatic essay grading, essay traits are important for scoring the essay holistically . a single-task learning system gives the best results for scoring essays holistically and scoring essay traits.
Approach:	They propose a way to score essays using a multi-task learning approach . they compare the MTL-based BiLSTM system to a single-task Learning approach based on LSTMs and BiLStms .
Outcome:	The proposed system gives better results for scoring essay holistically and scoring essay traits.

Copied to clipboard

Jianfei Yu, Jing Jiang, Ling Min Serena Khoo, Hai Leong Chieu, Rui Xia

Quality Estimation-Assisted Automatic Post-Editing (2023.findings-emnlp)

Challenge:	Existing approaches to rumor verification and stance classification fail to exploit intertask dependencies .
Approach:	They propose a Hierarchical Transformer model which uses BERT to obtain thread representations . they propose 'coupled' transformer modules to capture intertask interactions and a post-level attention layer to use predicted stance labels for RV.
Outcome:	The proposed model outperforms existing methods on two benchmark datasets.

Copied to clipboard

Sourabh Deoghare, Diptesh Kanojia, Fred Blain, Tharindu Ranasinghe, Pushpak Bhattacharyya

Machine Translation

Challenge:	Existing APE and QE combination strategies have not shown significant performance gains in the field of automatic post-editing (APE).
Approach:	They propose to train a model on APE and QE tasks to improve the APE performance by using a multi-task learning methodology that treats both tasks as a 'bargaining game' they also investigate various existing combination strategies and show that their approach achieves state-of-the-art performance for a ‘distant’ language pair, viz., English-Marathi.
Outcome:	The proposed model improves on two different language pairs, viz., English-Marathi and English-German.

A Gradually Soft Multi-Task and Data-Augmented Approach to Medical Question Understanding (2021.acl-long)

Copied to clipboard

Khalil Mrini, Franck Dernoncourt, Seunghyun Yoon, Trung Bui, Walter Chang, Emilia Farcas, Ndapa Nakashole

exBERT: Extending Pre-trained Models with Domain-specific Vocabulary Under Constrained Training Resources (2020.findings-emnlp)

Challenge:	Existing methods for medical question understanding often fail to provide high recall in answer retrieval.
Approach:	They propose a multi-task learning method with data augmentation for medical question understanding that uses just one dataset to optimize for both tasks.
Outcome:	The proposed method outperforms existing MTL methods across 4 datasets of medical question pairs in ROUGE scores, RQE accuracy and human evaluation.

Copied to clipboard

Wen Tai, H. T. Kung, Xin Dong, Marcus Comiter, Chang-Fu Kuo

Bridging Resolution: Making Sense of the State of the Art (2021.naacl-main)

Challenge:	Existing methods to train pre-trained models with limited corpus and computational resources are limited by the complexity of the training resources.
Approach:	They propose a method to extend BERT pre-trained models from a general domain to a new pre-train model for a specific domain with a different additive vocabulary.
Outcome:	The proposed method outperforms existing methods on biomedical benchmark tasks using the MTL-Bioinformatics-2016 dataset.

Copied to clipboard

Hideo Kobayashi, Vincent Ng

Information Extraction & Text Mining

Challenge:	bridging resolution is a task that involves identifying and resolving bridling/associative anaphors, which are anamorphic references to non-identical associated antecedents.
Approach:	They propose a hybrid rule-based and MTL approach that would enable a better understanding of their comparative strengths and weaknesses.
Outcome:	The proposed model would be able to better understand their strengths and weaknesses and perform a manual analysis of the errors made by the model.

Improving Gradient Trade-offs between Tasks in Multi-task Text Classification (2023.acl-long)

Copied to clipboard

Heyan Chai, Jinhao Cui, Ye Wang, Min Zhang, Binxing Fang, Qing Liao

Soft Representation Learning for Sparse Transfer (P19-1)

Challenge:	Existing methods to mitigate task conflict problem are heuristics or gradient-based algorithms to achieve an arbitrary Pareto optimal trade-off among different tasks .
Approach:	They propose a gradient trade-off approach to mitigate the task conflict problem by using heuristics or gradient-based algorithms to achieve an arbitrary Pareto optimal trade- off among different tasks.
Outcome:	The proposed model can achieve an arbitrary Pareto optimal trade-off among different tasks near the main objective of multi-task text classification (MTC) it is found that training all tasks simultaneously yields degraded performance than learning them independently, leading to poor training.

Copied to clipboard

Haeju Park, Jinyoung Yeo, Gengyu Wang, Seung-won Hwang

Representation Learning

Challenge:	Using adversarial training, we can “soft-code” shared and private spaces to avoid sparse sharing.
Approach:	They propose to use adversarial training to “soft-code” shared and private spaces to avoid the shared space gets too sparse.
Outcome:	The proposed architecture avoids sparse sharing of shared and private spaces, and also deals with low-quality input.

Exploring Logically Dependent Multi-task Learning with Causal Inference (2020.emnlp-main)

Copied to clipboard

Wenqing Chen, Jidong Tian, Liqiang Xiao, Hao He, Yaohui Jin

Dancing in Chains: Reconciling Instruction Following and Faithfulness in Language Models (2024.emnlp-main)

Challenge:	Hierarchical multi-task learning models can utilize task dependencies by stacking encoders and outperform democratic ones.
Approach:	They propose a model that utilizes the labels of all lower-level tasks and a Gumbel sampling model to deal with cascading errors.
Outcome:	The proposed model outperforms democratic models on six out of seven subtasks and achieves state-of-the-art on the two English and one Chinese datasets.

Copied to clipboard

Zhengxuan Wu, Yuhao Zhang, Peng Qi, Yumo Xu, Rujun Han, Yian Zhang, Jifan Chen, Bonan Min, Zhiheng Huang

SC-LSTM: Learning Task-Specific Representations in Multi-Task Learning for Sequence Labeling (N19-1)

Challenge:	Modern language models fail to follow human instructions while being faithful . a trade-off exists between instruction following and faithfulness when training LMs .
Approach:	They propose a method that relies on Reject-sampling by Self-instruct with Continued Fine-tuning to train LMs to follow human instructions while being faithful.
Outcome:	The proposed method outperforms vanilla MTL with high-quality data, but with significantly smaller data.

Copied to clipboard

Peng Lu, Ting Bai, Philippe Langlais

Multi-Task Learning for Sequence Tagging: An Empirical Study (C18-1)

Challenge:	Multi-task learning (MTL) has been studied for sequence labeling tasks . auxiliary tasks are selected specifically to improve performance of a target task .
Approach:	They propose a shared-cell long-short-term memory cell which contains shared parameters that can learn from all tasks and task-specific parameters that could learn task-related information.
Outcome:	The proposed model can learn from all tasks and task-specific parameters.

Copied to clipboard

Soravit Changpinyo, Hexiang Hu, Fei Sha

A Multi-Task Learning Framework for Modeling Engagement and Topic-Sensitive Responses in Arabic Women’s Discourse (2026.findings-eacl)

Challenge:	Existing work on "pairwise" MTL has been validated in sequence tagging but key issues remain about its effectiveness.
Approach:	They propose three general multi-task learning approaches on 11 sequence tagging tasks.
Outcome:	The proposed approaches improve on 11 sequence tagging tasks.

Copied to clipboard

Mabrouka Bessghaier, Md. Rafiul Biswas, Shimaa Ibrahim, Wajdi Zaghouani

Normalizing Compositional Structures Across Graphbanks (2020.coling-main)

Challenge:	a corpus of 158k arab Facebook posts spanning women's rights, gender debates, and economic empowerment reveals patterns of public opinion that vary dramatically across regional and cultural contexts.
Approach:	They propose a multi-task learning framework that learns audience reaction classification and engagement magnitude regression and non-engagement detection.
Outcome:	The proposed model achieves a test macro-F1 of 72.4 and weighted-F1. It measures 158k posts across gender issues, legal rights advocacy, gender identity discussions, and economic empowerment.

Copied to clipboard

Lucia Donatelli, Jonas Groschwitz, Matthias Lindemann, Alexander Koller, Pia Weißenhorn

Structured Data in NLP

Challenge:	Graph-based meaning representations (MRs) exhibit structural differences that reflect different theoretical and design considerations, presenting challenges to uniform linguistic analysis and cross-framework semantic parsing.
Approach:	They propose a method to normalize MRs at the compositional level by linguistically-grounded rules.
Outcome:	The proposed method increases the match in compositional structure between MRs and improves multi-task learning in a low-resource setting.

Estimating the influence of auxiliary tasks for multi-task learning of sequence tagging tasks (2020.acl-main)

Copied to clipboard

Fynn Schröder, Chris Biemann

MetaWeighting: Learning to Weight Tasks in Multi-Task Learning (2022.findings-acl)

Challenge:	Multitask learning and transfer learning are techniques to overcome data scarcity . finding suitable auxiliary datasets for multitask learning is a trial-and-error approach .
Approach:	They propose to automatically assess the similarity of sequence tagging datasets to identify beneficial auxiliary data for MTL or TL setups.
Outcome:	The proposed methods can compute similarity between two sequence tagging datasets . they show that the same measures correlate with the change in test score of the auxiliary dataset .

Copied to clipboard

Yuren Mao, Zekai Wang, Weiwei Liu, Xuemin Lin, Pengtao Xie

HLDC: Hindi Legal Documents Corpus (2022.findings-acl)

Challenge:	Existing task weighting methods assign weights only based on training losses, while ignoring the gap between the training loss and generalization loss.
Approach:	They propose a task weighting algorithm which automatically weights the tasks via a learning-to-learn paradigm and a multi-task text classification paradigm.
Outcome:	Extensive experiments show that the proposed method outperforms existing methods in multi-task text classification.

Copied to clipboard

Arnav Kapoor, Mudit Dhawan, Anmol Goel, Arjun T H, Akshala Bhatnagar, Vibhu Agrawal, Amul Agrawal, Arnab Bhattacharya, Ponnurangam Kumaraguru, Ashutosh Modi

Challenge:	Existing systems that process legal documents are lacking high-quality corpora in low resource languages such as Hindi.
Approach:	They propose a Hindi Legal Documents Corpus (HLDC) that contains 900K legal documents in Hindi.
Outcome:	The proposed model is based on a corpus of more than 900K legal documents in Hindi.

Towards Better Multi-task Learning: A Framework for Optimizing Dataset Combinations in Large Language Models (2025.findings-naacl)

Copied to clipboard

Zaifu Zhan, Rui Zhang

Syntax-Aware Opinion Role Labeling with Dependency Graph Convolutional Networks (2020.acl-main)

Challenge:	Using a neural network, large language models can be trained on multiple tasks, allowing them to perform tasks efficiently.
Approach:	They propose a framework that leverages a neural network to select the best dataset combinations for enhancing multi-task learning (MTL) They propose to iteratively refine the selection, greatly improving efficiency while being model-, dataset-, and domain-independent.
Outcome:	The proposed framework iteratively refines the selection, greatly improving efficiency, while being model-, dataset-, and domain-independent.

Copied to clipboard

Bo Zhang, Yue Zhang, Rui Wang, Zhenghua Li, Min Zhang

Multimodality

Challenge:	Opinion role labeling (ORL) is a fine-grained opinion analysis task . due to the scarcity of labeled data, ORL remains challenging for data-driven methods due to its complexity and complexity.
Approach:	They propose to integrate syntactic knowledge into ORL models by comparing and integrating different representations and using dependency graph convolutional networks to encode parser information at different processing levels.
Outcome:	The proposed model achieves 4.34 higher F1 score than the current state-of-the-art.

“We will Reduce Taxes” - Identifying Election Pledges with Language Models (2021.findings-acl)

Copied to clipboard

Tommaso Fornaciari, Dirk Hovy, Elin Naurin, Julia Runeson, Robert Thomson, Pankaj Adhikari

Annotations Matter: Leveraging Multi-task Learning to Parse UD and SUD (2021.findings-acl)

Challenge:	a political party's manifestos are published before any election, but do they follow through? a new study uses neural models to distinguish between actual pledges and general statements .
Approach:	They use election manifestos of Swedish and Indian political parties to learn neural models that distinguish actual pledges from generic positions.
Outcome:	The proposed model can predict election year and manifesto's party, while context introduces noise.

Copied to clipboard

Zeeshan Ali Sayyed, Daniel Dakota

AutoSeM: Automatic Task Selection and Mixing in Multi-Task Learning (N19-1)

Challenge:	Multi-task learning (MTL) has shown promise in various NLP tasks such as semantic dependency parsing.
Approach:	They propose to use two parallel treebanks to improve parsing performance.
Outcome:	The proposed model is based on two parallel treebanks with similar annotation schemes but differing in linguistic annotation preferences.

Copied to clipboard

Han Guo, Ramakanth Pasunuru, Mohit Bansal

When Does Aggregating Multiple Skills with Multi-Task Learning Work? A Case Study in Financial NLP (2023.acl-long)

Challenge:	Multi-task learning is an inductive transfer mechanism that leverages information from related tasks to improve the primary model's generalization performance.
Approach:	They propose a multitask learning pipeline that finds relevant auxiliary tasks and learns their mixing ratio.
Outcome:	The proposed model can find relevant auxiliary tasks and learn their mixing ratio . the proposed model achieves significant performance boosts on several primary tasks .

Copied to clipboard

Jingwei Ni, Zhijing Jin, Qian Wang, Mrinmaya Sachan, Markus Leippold

BanditMTL: Bandit-based Multi-task Learning for Text Classification (2021.acl-long)

Challenge:	Multi-task learning (MTL) is a machine learning paradigm where multiple learning tasks are optimized simultaneously, exploiting commonalities and differences across them.
Approach:	They propose a parameter-efficient MTL architecture to improve task aggregation and to include loosely related skills from multiple datasets.
Outcome:	The proposed architecture outperforms single-task learning (STL) and is expected to outperformed it.

Copied to clipboard

Yuren Mao, Zekai Wang, Weiwei Liu, Xuemin Lin, Wenbin Hu

The Stem Cell Hypothesis: Dilemma behind Multi-Task Learning with Transformer Encoders (2021.emnlp-main)

Challenge:	Existing methods to regularize task variance are unexplored in multi-task text classification.
Approach:	They propose a multi-task learning method based on adversarial multi-armed bandit to regularize the task variance by means of a mirror gradient ascent-descent algorithm.
Outcome:	The proposed method achieves state-of-the-art in multi-task text classification.

Copied to clipboard

Han He, Jinho D. Choi

GradTS: A Gradient-Based Automatic Auxiliary Task Selection Method Based on Transformer Networks (2021.emnlp-main)

Challenge:	Multi-task learning with transformer encoders (MTL) has emerged as a powerful technique to improve performance on closely-related tasks for both accuracy and efficiency.
Approach:	They propose a multi-task learning technique that uses transformer encoders to improve performance on closely-related tasks.
Outcome:	The proposed method performs better on five NLP tasks than single-task learning on similar tasks.

Copied to clipboard

Weicheng Ma, Renze Lou, Kai Zhang, Lili Wang, Soroush Vosoughi

CoBa: Convergence Balancer for Multitask Finetuning of Large Language Models (2024.emnlp-main)

Challenge:	A key problem in multi-task learning (MTL) research is how to select high-quality auxiliary tasks automatically.
Approach:	They propose an automatic auxiliary task selection method based on gradient calculation in Transformer-based models that improves MT-DNN performance.
Outcome:	The proposed method improves MT-DNN performance on 8 natural language understanding (GLUE) tasks, while costing less than AUTOSEM and comparable GPU consumption.

Copied to clipboard

Zi Gong, Hang Yu, Cong Liao, Bingchang Liu, Chaoyu Chen, Jianguo Li

monoQA: Multi-Task Learning of Reranking and Answer Extraction for Open-Retrieval Conversational Question Answering (2022.emnlp-main)

Challenge:	Existing multi-task learning approaches for large language models fall short due to computational intensive or lack of simultaneous task convergence.
Approach:	They propose a new multi-task learning approach that dynamically adjusts task weights during the training process, ensuring that the validation loss of all tasks progresses towards convergence at an even pace.
Outcome:	The proposed approach improves the performance of large language models by up to 13% compared to the second-best approaches.

Copied to clipboard

Sarawoot Kongyoung, Craig Macdonald, Iadh Ounis

Natural Language Interfaces

Challenge:	Existing approaches to the Conversational Question Answering task have used multi-task learning to solve the task.
Approach:	They propose to use multi-task learning to improve the ORConvQA task by sharing the reranker and reader’s learned structure in a generative model.
Outcome:	The proposed model outperforms baseline models on the OR-QuAC and OR-CoQA datasets and significantly outperformed existing strong baseline models.

Sequence Labeling Parsing by Learning across Representations (P19-1)

Copied to clipboard

Michalina Strzyz, David Vilares, Carlos Gómez-Rodríguez

Representation Learning

Challenge:	Constituency and dependency parsing are the main abstractions for representing syntactic structure of sentences . constituency parsers are considered disjointed tasks, and their improvements have been obtained separately.
Approach:	They propose to add auxiliary loss to constituency parsing paradigms and explore a model that parses both paradigms at no cost.
Outcome:	The proposed model outperforms single-task models by 1.05 F1 points and 0.62 UAS points for constituency parsing and dependency parsers.

A Syntax-aware Multi-task Learning Framework for Chinese Semantic Role Labeling (D19-1)

Copied to clipboard

Qingrong Xia, Zhenghua Li, Min Zhang

Facilitating Cognitive Accessibility with LLMs: A Multi-Task Approach to Easy-to-Read Text Generation (2025.emnlp-main)

Challenge:	Semantic role labeling (SRL) aims to identify the predicate-argument structure of a sentence.
Approach:	They propose to use a unified span-based model for Chinese SRL as a strong baseline.
Outcome:	The proposed framework achieves state-of-the-art 87.54 and 88.5 F1 scores on the Chinese Proposition Bank and CoNLL-2009 datasets.

Copied to clipboard

François Ledoyen, Gaël Dias, Jeremie Pantin, Alexis Lechervy, Fabrice Maurel, Youssef Chahir

Joint Alignment of Multi-Task Feature and Label Spaces for Emotion Cause Pair Extraction (2022.coling-1)

Challenge:	Existing approaches to make complex texts more accessible for people with cognitive impairments are time-consuming and resource-intensive.
Approach:	They propose a multi-task learning approach that trains models jointly on text summarization, text simplification, and ETR generation.
Outcome:	The proposed approach outperforms other approaches in in-domain settings while achieving better generalization in out-of-domain scenarios.

Copied to clipboard

Shunjie Chen, Xiaochuan Shi, Jingye Li, Shengqiong Wu, Hao Fei, Fei Li, Donghong Ji

A Transformer-based Threshold-Free Framework for Multi-Intent NLU (2022.coling-1)

Challenge:	Existing methods for ECPE fail to model specific features and interactive features in between, or suffer from inconsistency of label prediction.
Approach:	They propose to align ECPE with a feature-task alignment mechanism to model emotion-&cause-specific features and the shared interactive feature.
Outcome:	The proposed model outperforms existing systems on all ECA subtasks.

Copied to clipboard

Lisung Chen, Nuo Chen, Yuexian Zou, Yong Wang, Xinzhong Sun

ScaLearn: Simple and Highly Parameter-Efficient Task Transfer by Learning to Scale (2024.findings-acl)

Challenge:	Existing models for multi-intent natural language understanding mainly detect multiple intents on threshold settings.
Approach:	They propose a transformer-based multi-intent NLU model with multi-task learning that exploits the information of the number of multiple intents in each utterance without additional manual annotations.
Outcome:	The proposed model achieves superior results on two public multi-intent datasets.

Copied to clipboard

Markus Frohmann, Carolin Holtermann, Shahed Masoudian, Anne Lauscher, Navid Rekabsaz

AdapterShare: Task Correlation Modeling with Adapter Differentiation (2022.emnlp-main)

Challenge:	Multi-task learning (MTL) has shown significant practical benefits when using language models . current two stage MTL introduces a substantial number of additional parameters .
Approach:	They propose a multi-task learning method that leverages existing knowledge for a target task.
Outcome:	The proposed method outperforms baselines on three benchmarks and two encoder LMs with a small number of transfer parameters.

Copied to clipboard

Zhi Chen, Bei Chen, Lu Chen, Kai Yu, Jian-Guang Lou

Green & Sustainable NLP

Challenge:	AdapterShare is an adapter differentiation method to explicitly model the task correlation among multiple tasks.
Approach:	They propose an adapter differentiation method to explicitly model the task correlation among multiple tasks.
Outcome:	The proposed method achieves 1.90 points improvement on five dialogue understanding tasks and 2.33 points gain on NLU tasks.

Improving Chinese Named Entity Recognition with Multi-grained Words and Part-of-Speech Tags via Joint Modeling (2024.lrec-main)

Copied to clipboard

Chenhui Dou, Chen Gong, Zhenghua Li, Zhefeng Wang, Baoxing Huai, Min Zhang

Syntactic Text Processing

Challenge:	Named entity recognition (CNER) is a fundamental task in natural language processing (NLP).
Approach:	They propose a tree parsing approach for jointly modeling Chinese named entity recognition (CNER) with multi-grained word segmentation (MWS) and POS tagging tasks.
Outcome:	The proposed approach achieves better or comparable performance with current methods.

Multi-Task Learning of Query Generation and Classification for Generative Conversational Question Rewriting (2023.findings-emnlp)

Copied to clipboard

Sarawoot Kongyoung, Craig MacDonald, Iadh Ounis

Information Retrieval

Challenge:	Existing approaches to identifying ambiguous questions as part of a conversation have not addressed this challenge.
Approach:	They propose a multi-task learning approach that uses a text generation model for question rewriting and classification.
Outcome:	The proposed approach outperforms single-task learning baselines on three LIF test sets.

MAGPIE: Multi-Task Analysis of Media-Bias Generalization with Pre-Trained Identification of Expressions (2024.lrec-main)

Copied to clipboard

Tomáš Horych, Martin Paul Wessel, Jan Philip Wahle, Terry Ruas, Jerome Waßmuth, André Greiner-Petter, Akiko Aizawa, Bela Gipp, Timo Spinde

Fine-Tuning Language Models on Multiple Datasets for Citation Intention Classification (2024.findings-emnlp)

Challenge:	Existing approaches to media bias detection lack generalizability, resulting in limited generalizarability.
Approach:	They propose a large-scale multi-task pre-training approach specifically tailored for media bias detection that can be used to train 59 bias-related tasks.
Outcome:	The proposed approach outperforms existing methods on the BABE dataset with a relative improvement of 3.3% F1-score.

Copied to clipboard

Zeren Shui, Petros Karypis, Daniel Karls, Mingjian Wen, Saurav Manchanda, Ellad Tadmor, George Karypis

Impartial Multi-task Representation Learning via Variance-invariant Probabilistic Decoding (2025.acl-long)

Challenge:	Prior research has shown that pretrained language models (PLMs) can achieve state-of-the-art performance on CIC benchmarks.
Approach:	They propose a multi-task learning framework that fine-tunes pretrained language models on a dataset of primary interest together with multiple auxiliary CIC datasets to take advantage of additional supervision signals.
Outcome:	The proposed framework outperforms current state-of-the-art models on small datasets while aligning with the best-performing model on a large dataset.

Copied to clipboard

Dou Hu, Lingwei Wei, Wei Zhou, Songlin Hu

Revealing the impact of synthetic native samples and multi-tasking strategies in Hindi-English code-mixed humour and sarcasm detection (2025.findings-emnlp)

Challenge:	Existing methods focus on balancing loss or gradients but fail to address this issue due to the representation discrepancy in latent space.
Approach:	They propose a framework that harmonizes representation spaces across tasks to ensure impartial learning by harmonizing representation spaces.
Outcome:	The proposed framework outperforms 12 representative methods under the same multi-task settings, especially in heterogeneous task combinations and data-constrained scenarios.

Copied to clipboard

Debajyoti Mazumder, Aakash Kumar, Jasabanta Patro

A Learnable Skill Combination Strategy for Multi-task Learning in Natural Language Understanding (2026.findings-acl)

Challenge:	Specifically, we tried native sample mixing, multi-task learning, and prompting and instruction finetuning very large multilingual language models (VMLMs).
Approach:	They used native sample mixing, multi-task learning and prompting and instruction finetuning to improve code-mixed humour and sarcasm detection.
Outcome:	The proposed methods improve humour and sarcasm detection by adding native samples to training sets and multitask learning and prompting and instruction finetuning VMLMs.

Copied to clipboard

Zhe Yang, Yi Huang, Yaqin Chen, Mengfei Guo, Xiaoting Wu, Junlan Feng

Multi-Task Representation Alignment on Language Understanding: A Mutual Information Perspective (2026.acl-long)

Challenge:	a novel multi-task learning framework for domain-specific natural language understanding tasks addresses these limitations by combing multiple tasks into a single framework.
Approach:	They propose a multi-task learning framework that decomposes the language model into modular skill components and employs a dynamic, learnable skill-combination mechanism to adaptively handle diverse tasks.
Outcome:	The proposed framework surpasses conventional multi-task learning approaches in performance.

Copied to clipboard

Dou Hu, Lingwei Wei, Hongjiang Xiao, Songlin Hu, Yuan Zhang