Know-Adapter: Towards Knowledge-Aware Parameter-Efficient Transfer Learning for Few-shot Named Entity Recognition (2024.lrec-main)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is a fundamental task in natural language processing. |
| Approach: | They propose a knowledgeable adapter to incorporate structure and semantic knowledge of knowledge graphs into PLMs for few-shot NER. |
| Outcome: | The proposed adapter improves the quality of retrieved information by adding explicit knowledge from external sources to PEFTs. |
Similar Papers
KG-Adapter: Enabling Knowledge Graph Integration in Large Language Models through Parameter-Efficient Fine-Tuning (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are criticized for lack of expertise and knowledge conflict . KG-Adapter is a parameter-level KG integration method for decoder-only LLMs . |
| Approach: | They propose a parameter-level KG integration method based on parameter-efficient fine-tuning . they use KG-Adapter to integrate knowledge graphs with LLMs and perform joint reasoning . |
| Outcome: | The proposed method outperforms the current state-of-the-art method on four datasets for two different tasks. |
Hit the Nail on the Head: Parameter-Efficient Multi-task Tuning via Human Language Intervention (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies show that PEFT on small pre-trained language models improves multitasking capabilities. |
| Approach: | They propose a multi-task learning framework that enables transfer of prior knowledge across tasks . they attach task descriptions to input samples and map them to task embeddings . |
| Outcome: | The proposed method improves performance on a T5 model and in decoder-only models . |
Empowering parameter-efficient transfer learning by recognizing the kernel structure in self-attention (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to fine-tune pre-trained language models are parameter efficient . fine- tuning the models requires multiple copies of the parameters, which is inefficient. |
| Approach: | They propose to use kernel-based adapters to tune only a few parameters while freezing the rest of the parameters. |
| Outcome: | The proposed methods achieve or improve strong performance over a diverse set of natural language generation and understanding tasks. |
Context-Aware Adapter Tuning for Few-Shot Relation Learning in Knowledge Graphs (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to predict instances for missing relations on knowledge graphs are limited by their limited training examples. |
| Approach: | They propose a context-aware adapter for few-shot relation learning in KGs . they propose tunable relation adaptation and contextual information for each relation . |
| Outcome: | Experiments on three benchmark KGs validate the superiority of RelAdapter over state-of-the-art methods. |
KnowLA: Enhancing Parameter-efficient Finetuning with Knowledgeable Adaptation (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for parameter-efficient finetuning (PEFT) are limited and only finetune a small number of parameters using limited instruction data. |
| Approach: | They propose a method that inserts an adaptation layer into an LLM to integrate embeddings of entities appearing in the input text. |
| Outcome: | The proposed method can activate parameterized knowledge in an LLM without changing its parameters or input prompts. |
K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters (2021.findings-acl)
Copied to clipboard
Ruize Wang, Duyu Tang, Nan Duan, Zhongyu Wei, Xuanjing Huang, Jianshu Ji, Guihong Cao, Daxin Jiang, Ming Zhou
| Challenge: | Existing methods for injecting knowledge into pre-trained models are inconsistent and can flush out knowledge when multiple kinds of knowledge are injected. |
| Approach: | They propose a framework that retains the original parameters of pre-trained models fixed and supports the development of versatile knowledge-infused models. |
| Outcome: | The proposed framework retains the original parameters of the pre-trained model fixed and supports the development of versatile knowledge-infused models. |
Adapters: A Unified Library for Parameter-Efficient and Modular Transfer Learning (2023.emnlp-demo)
Copied to clipboard
Clifton Poth, Hannah Sterz, Indraneil Paul, Sukannya Purkayastha, Leon Engländer, Timo Imhof, Ivan Vulić, Sebastian Ruder, Iryna Gurevych, Jonas Pfeiffer
| Challenge: | Adapters is an open-source library that unifies parameter-efficient and modular transfer learning in large language models. |
| Approach: | They propose to integrate 10 different methods into a unified interface for parameter-efficient and modular transfer learning in large language models. |
| Outcome: | The proposed library is able to perform on multiple NLP tasks and is open-source. |
LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models (2023.emnlp-main)
Copied to clipboard
Zhiqiang Hu, Lei Wang, Yihuai Lan, Wanyu Xu, Ee-Peng Lim, Lidong Bing, Xing Xu, Soujanya Poria, Roy Lee
| Challenge: | Large language models (LLMs) have shown unprecedented performance across various tasks. |
| Approach: | They propose an easy-to-use framework that integrates adapters into LLMs . they evaluate adapters on 14 datasets from two different reasoning tasks . |
| Outcome: | The proposed framework can be used to fine-tune open-access language models with task-specific data and instruction data. |
AdapterHub Playground: Simple and Flexible Few-Shot Learning with Adapters (2022.acl-demo)
Copied to clipboard
Tilman Beck, Bela Bohlender, Christina Viehmann, Vincent Hane, Yanik Adamson, Jaber Khuri, Jonas Brossmann, Jonas Pfeiffer, Iryna Gurevych
| Challenge: | AdapterHub Playground is an open-access tool for researchers to use pretrained language models without writing a single line of code. |
| Approach: | They propose a tool which allows researchers to leverage pretrained models without writing a single line of code for a variety of NLP tasks. |
| Outcome: | The proposed model can be used for prediction, training and analysis of textual data without writing a single line of code. |
Parameter-Efficient Domain Knowledge Integration from Multiple Sources for Biomedical Pre-trained Language Models (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing domain-specific pre-trained language models (PLMs) rely on self-supervised learning over large amounts of domain text, without explicitly integrating domain- specific knowledge. |
| Approach: | They propose to integrate domain knowledge from diverse sources into PLMs by using adapters that are pre-trained for individual domain knowledge sources and integrated via an attention-based knowledge controller. |
| Outcome: | The proposed architecture integrates domain knowledge from diverse sources into PLMs in a parameter-efficient way. |