Multi-Task Representation Alignment on Language Understanding: A Mutual Information Perspective (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to multitask learning fail to address task interference issues . Existing methods focus on task balancing or probabilistic modeling but fail to learn sufficient representations for all target tasks. |
| Approach: | They propose a multi-task representation alignment framework to achieve task-specific alignment and self-alignment on shared representations from a mutual information perspective. |
| Outcome: | The proposed framework outperforms 13 representative MTL methods under label-noisy and data-constrained conditions. |
Similar Papers
Impartial Multi-task Representation Learning via Variance-invariant Probabilistic Decoding (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods focus on balancing loss or gradients but fail to address this issue due to the representation discrepancy in latent space. |
| Approach: | They propose a framework that harmonizes representation spaces across tasks to ensure impartial learning by harmonizing representation spaces. |
| Outcome: | The proposed framework outperforms 12 representative methods under the same multi-task settings, especially in heterogeneous task combinations and data-constrained scenarios. |
Adaptive Knowledge Sharing in Multi-Task Learning: Improving Low-Resource Neural Machine Translation (P18-2)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) requires large amounts of bilingual data to learn a translation model with reasonable quality. |
| Approach: | They propose to extend recurrent units with multiple "blocks" along with a trainable "routing network" this allows for adaptive collaboration by dynamic sharing of blocks conditioned on the task at hand, input, and model state. |
| Outcome: | Empirical evaluations of two low-resource translation tasks show +1 BLEU score improvements compared to strong baselines. |
A Learnable Skill Combination Strategy for Multi-task Learning in Natural Language Understanding (2026.findings-acl)
Copied to clipboard
| Challenge: | a novel multi-task learning framework for domain-specific natural language understanding tasks addresses these limitations by combing multiple tasks into a single framework. |
| Approach: | They propose a multi-task learning framework that decomposes the language model into modular skill components and employs a dynamic, learnable skill-combination mechanism to adaptively handle diverse tasks. |
| Outcome: | The proposed framework surpasses conventional multi-task learning approaches in performance. |
Bag-of-Words Transfer: Non-Contextual Techniques for Multi-Task Learning (D19-61)
Copied to clipboard
| Challenge: | Existing approaches to multi-task learning take advantage of transfer among tasks . generative reconstruction of the observations is not included in the standard framework . |
| Approach: | They propose to use a syntactically-oblivious pooling encoder and pre-trained word embeddings to improve sentence-level representations. |
| Outcome: | The proposed techniques yield similar performance on a universe of task combinations while reducing training time and model size. |
A Survey of Multi-task Learning in Natural Language Processing: Regarding Task Relatedness and Training Methods (2023.eacl-main)
Copied to clipboard
| Challenge: | Multi-task learning is a popular approach in natural language processing because of its commonalities and differences. |
| Approach: | They propose to summarize recent advances in multi-task learning methods based on their task relatedness into two general multi-step training methods. |
| Outcome: | The proposed methods summarize the tasks and discuss future directions. |
SC-LSTM: Learning Task-Specific Representations in Multi-Task Learning for Sequence Labeling (N19-1)
Copied to clipboard
| Challenge: | Multi-task learning (MTL) has been studied for sequence labeling tasks . auxiliary tasks are selected specifically to improve performance of a target task . |
| Approach: | They propose a shared-cell long-short-term memory cell which contains shared parameters that can learn from all tasks and task-specific parameters that could learn task-related information. |
| Outcome: | The proposed model can learn from all tasks and task-specific parameters. |
Multi-Task Learning for Sequence Tagging: An Empirical Study (C18-1)
Copied to clipboard
| Challenge: | Existing work on "pairwise" MTL has been validated in sequence tagging but key issues remain about its effectiveness. |
| Approach: | They propose three general multi-task learning approaches on 11 sequence tagging tasks. |
| Outcome: | The proposed approaches improve on 11 sequence tagging tasks. |
Analyzing Text Representations by Measuring Task Alignment (2023.acl-short)
Copied to clipboard
| Challenge: | Recent advances in text classification have shown that pre-trained representations are key for text classification. |
| Approach: | They propose a task alignment score that measures alignment at different levels of granularity. |
| Outcome: | The proposed score shows that task alignment can explain the performance of a given representation. |
HMCL: Task-Optimal Text Representation Adaptation through Hierarchical Contrastive Learning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Hierarchical Multilevel Contrastive Learning (HMCL) improves text representation for general large language models. |
| Approach: | a new contrastive learning framework is developed to improve general large language models . HMCL integrates 3-level semantic differentiation and unifies contrastive and pair classification into a strategy . |
| Outcome: | HMCL outperforms unsupervised methods and supervised fine-tuning approaches in multi-domain and multilingual benchmarks. |
Middle-Layer Representation Alignment for Cross-Lingual Transfer in Fine-Tuned LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Effective cross-lingual transfer is hindered by performance gaps and the scarcity of fine-tuning data in many languages. |
| Approach: | They propose a middle-layer alignment objective integrated into task-specific training to improve cross-lingual transfer across languages. |
| Outcome: | The proposed method improves cross-lingual transfer to lower-resource languages and can be merged with existing modules without full re-training. |