TEAM: A multitask learning based Taxonomy Expansion approach for Attach and Merge (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for automating taxonomy expansion are attach and merge . elucidating the problem of limited coverage of WordNets is presented . |
| Approach: | They propose a multitask learning-based deep learning method that performs both merge and attach operations in a single model. |
| Outcome: | The proposed method outperforms state-of-the-art models on three WordNet taxonomies . it performs both merge and attach operations and also provides encouraging performance for merge operation . |
Similar Papers
Training-free LLM Merging for Multi-task Learning (2025.acl-long)
Copied to clipboard
Zichuan Fu, Xian Wu, Yejing Wang, Wanyu Wang, Shanshan Ye, Hongzhi Yin, Yi Chang, Yefeng Zheng, Xiangyu Zhao
| Challenge: | Large Language Models (LLMs) have demonstrated exceptional capabilities across diverse natural language processing tasks. |
| Approach: | They propose a training-free method for unifying different specialized LLMs into a single model using model-wise and layer-wise pruning and scaling. |
| Outcome: | The proposed method outperforms existing merging techniques and surpasses models fine-tuned on combined datasets in most scenarios. |
Selecting and Merging: Towards Adaptable and Scalable Named Entity Recognition with Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to align large language models with information extraction tasks are costly and not all training data benefits target domains. |
| Approach: | They propose a framework which dynamically Selects and Merges expert models at inference time and combines experts beneficial to target domains. |
| Outcome: | The proposed framework outperforms the unified model by 10% on multiple benchmarks. |
Multitask Learning-Based Neural Bridging Reference Resolution (2020.coling-main)
Copied to clipboard
| Challenge: | Existing models for bridging references lack large corpora annotated with briding references . second challenge is different definitions of bridding used in different corpors . |
| Approach: | They propose a multi-task learning-based neural model for bridging reference resolution . they show substantial improvements of up to 8 p.p. on full briding resolution compared to previous models . |
| Outcome: | The proposed model outperforms the best reported results on full bridging resolution by up to 8 p.p. |
A supervised approach to taxonomy extraction using word embeddings (L18-1)
Copied to clipboard
| Challenge: | a recent evaluation of a method for organizing texts into a hierarchy showed that it did not outperform a baseline. |
| Approach: | They propose a method that uses supervised learning to combine multiple features with a support vector machine classifier including the baseline features. |
| Outcome: | The proposed method outperforms the baseline method and provides stronger method for identifying taxonomic relations than previous methods. |
Recurrent Interaction Network for Jointly Extracting Entities and Classifying Relations (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to solve the extraction problem learn interactions between the two tasks through a shared network . |
| Approach: | They propose to use multi-task learning to address the joint extraction of entity and relation . they exploit correlation between ER and relation classification tasks to improve performance . |
| Outcome: | Empirical results show that the proposed model improves on two real-world datasets. |
Insert or Attach: Taxonomy Completion via Box Embedding (2024.acl-long)
Copied to clipboard
| Challenge: | Existing taxonomy expansion methods embed concepts as vectors in Euclidean space, causing incorrectly model asymmetric relations. |
| Approach: | They propose to use box containment and center closeness to create geometric scorers that capture intrinsic relationships between concepts. |
| Outcome: | The proposed framework outperforms existing methods on four real-world datasets. |
Arcee’s MergeKit: A Toolkit for Merging Large Language Models (2024.emnlp-industry)
Copied to clipboard
Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi, Luke Meyers, Vladimir Karpukhin, Brian Benedict, Mark McQuade, Jacob Solawetz
| Challenge: | Open-source language models can merge their parameters to improve performance and versatility without additional training. |
| Approach: | They propose to integrate model checkpoints into powerful multitask models without additional training. |
| Outcome: | the library has facilitated the merging of thousands of models, contributing to some of the world’s most powerful open-source model checkpoints. |
Exploiting Entity BIO Tag Embeddings and Multi-task Learning for Relation Extraction with Imbalanced Data (P19-1)
Copied to clipboard
| Challenge: | Existing methods to perform relation extraction are feature-based or kernel-based, but the results of our study show that they can improve the performance of a baseline model with more than 10% absolute increase in F1-score. |
| Approach: | They propose a multi-task architecture which jointly trains a model to perform relation identification with cross-entropy loss and relation classification with ranking loss. |
| Outcome: | The proposed model outperforms the state-of-the-art models on ACE 2005 Chinese and English corpus and significantly improves the performance of a baseline model with more than 10% increase in F1-score. |
Mergenetic: a Simple Evolutionary Model Merging Library (2025.acl-demo)
Copied to clipboard
| Challenge: | Recent work shows that combining model merging with evolutionary algorithms can boost performance, but there is currently no library for experimenting with different evolutionary algorithms and merging methods. |
| Approach: | They propose an open-source library for evolutionary model merging that enables easy composition of merging methods and evolutionary algorithms while incorporating lightweight fitness estimators to reduce evaluation costs. |
| Outcome: | The proposed library produces competitive results across languages and tasks using modest hardware. |
Studying Taxonomy Enrichment on Diachronic WordNet Versions (2020.coling-main)
Copied to clipboard
| Challenge: | Ontologies, taxonomies and thesauri are used in many NLP tasks but are often not maintained. |
| Approach: | They propose methods for taxonomy enrichment in a resource-poor setting . they also create novel datasets for training and evaluating taxonomies . |
| Outcome: | The proposed methods are applicable to English and Russian datasets and can be used in other languages. |