Selecting and Merging: Towards Adaptable and Scalable Named Entity Recognition with Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to align large language models with information extraction tasks are costly and not all training data benefits target domains. |
| Approach: | They propose a framework which dynamically Selects and Merges expert models at inference time and combines experts beneficial to target domains. |
| Outcome: | The proposed framework outperforms the unified model by 10% on multiple benchmarks. |
Similar Papers
Split-Merge: Scalable and Memory-Efficient Merging of Expert LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | a zero-shot merging framework for large language models consolidates specialized domain experts into a single model without any further training. |
| Approach: | They propose a zero-shot merging framework that consolidates specialized domain experts into a single model without further training. |
| Outcome: | Experiments on code generation, mathematical reasoning, medical question answering, and instruction-following benchmarks confirm the versatility and effectiveness of the proposed framework. |
ProgGen: Generating Named Entity Recognition Datasets Step-by-step with Self-Reflexive Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) exhibit remarkable adaptability across domains, but they are often not suitable for structured knowledge extraction tasks such as named entity recognition (NER). |
| Approach: | They propose a method that instructs LLMs to self-reflect on the specific domain and generates domain-relevant attributes for creating attribute-rich training data. |
| Outcome: | The proposed method produces NER datasets in domains with domain-relevant attributes and generates entity terms and NER context data around these entities. |
Beyond Boundaries: Learning a Universal Entity Taxonomy across Datasets and Languages for Open Named Entity Recognition (2025.coling-main)
Copied to clipboard
Yuming Yang, Wantong Zhao, Caishuang Huang, Junjie Ye, Xiao Wang, Huiyuan Zheng, Yang Nan, Yuran Wang, Xueying Xu, Kaixin Huang, Yunke Zhang, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Current Large Language Models struggle with complex entity taxonomies in open domains and lack NER capabilities. |
| Approach: | They propose a dataset to guide LLMs' generalization in Open NER under a universal entity taxonomy. |
| Outcome: | The proposed model outperforms GPT-4 in 3 out-of-domain benchmarks across 15 datasets and 6 languages. |
GPT-NER: Named Entity Recognition via Large Language Models (2025.findings-naacl)
Copied to clipboard
Shuhe Wang, Xiaofei Sun, Xiaoya Li, Rongbin Ouyang, Fei Wu, Tianwei Zhang, Jiwei Li, Guoyin Wang, Chen Guo
| Challenge: | Large-scale language models (LLMs) have shown impressive ability for in-context learning with limited training data. |
| Approach: | They propose a novel sequence labeling task that transforms a sequence labeled as a text-generation task into a self-verification task that LLMs can adapt to. |
| Outcome: | The proposed model performs better on NER than supervised models on a variety of tasks . the proposed model can be easily adapted by LLMs to generate a text sequence . |
Mitigating Training Imbalance in LLM Fine-Tuning via Selective Parameter Merging (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies suggest that the order of training samples can affect model performance, but this is not the case. |
| Approach: | They propose to merge supervised fine-tuning models with different data orders to mitigate this imbalance by parameter merging. |
| Outcome: | The proposed method outperforms the weighted-average method on five datasets. |
MergeME: Model Merging Techniques for Homogeneous and Heterogeneous MoEs (2025.naacl-long)
Copied to clipboard
Yuhang Zhou, Giannis Karamanolakis, Victor Soto, Anna Rumshisky, Mayank Kulkarni, Furong Huang, Wei Ai, Jianhua Lu
| Challenge: | State-of-the-art methods for merging expert models with different architectures do not address parameter interference and require extensive fine-tuning to restore performance. |
| Approach: | They propose a method for merging experts with different architectures into a unified Mixture-of-Experts model with a goal of enhancing performance in each domain while retaining effectiveness on general tasks. |
| Outcome: | Experiments across multiple domains show that the proposed methods reduce fine-tuning costs and improve performance over state-of-the-art methods. |
Semi-supervised Fine-tuning for Large Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing LLMs require labeled data, which can be costly in real-world applications. |
| Approach: | They propose a framework that can fully exploit labeled and unlabeled data for LLM fine-tuning . they conducted experiments using GPT-4o-mini and Llama-3.1 on seven general or domain-specific datasets . |
| Outcome: | The proposed framework can fully exploit labeled and unlabeled data for LLM alignment from a propagate-and-select manner. |
Unlocking the Potential of Model Merging for Low-Resource Languages (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Adapting large language models (LLMs) to new languages requires continual pre-training followed by supervised fine-tuning. |
| Approach: | They propose a model merging solution that integrates LLMs with distinct capabilities into a single model without additional training. |
| Outcome: | The proposed model merging outperforms CT-then-SFT in low-resource languages with scarce data. |
Dynamic Fisher-weighted Model Merging via Bayesian Optimization (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing merging approaches involve scaling the parameters model-wise or integrating parameter importance parameter-wise. |
| Approach: | They propose a method for merging model-based models at the parameter level without training data or joint training. |
| Outcome: | The proposed model merging framework outperforms baseline models on validation sets. |
Contextual Augmentation for Entity Linking using Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Entity Linking involves detecting and linking entity mentions in natural language texts to a knowledge graph. |
| Approach: | They propose a fine-tuned model that integrates entity recognition and disambiguation in a unified framework. |
| Outcome: | The proposed model achieves state-of-the-art on out-of domain datasets and compares with baselines. |