AdapterHub Playground: Simple and Flexible Few-Shot Learning with Adapters (2022.acl-demo)
Copied to clipboard
Tilman Beck, Bela Bohlender, Christina Viehmann, Vincent Hane, Yanik Adamson, Jaber Khuri, Jonas Brossmann, Jonas Pfeiffer, Iryna Gurevych
| Challenge: | AdapterHub Playground is an open-access tool for researchers to use pretrained language models without writing a single line of code. |
| Approach: | They propose a tool which allows researchers to leverage pretrained models without writing a single line of code for a variety of NLP tasks. |
| Outcome: | The proposed model can be used for prediction, training and analysis of textual data without writing a single line of code. |
Similar Papers
Adapters: A Unified Library for Parameter-Efficient and Modular Transfer Learning (2023.emnlp-demo)
Copied to clipboard
Clifton Poth, Hannah Sterz, Indraneil Paul, Sukannya Purkayastha, Leon Engländer, Timo Imhof, Ivan Vulić, Sebastian Ruder, Iryna Gurevych, Jonas Pfeiffer
| Challenge: | Adapters is an open-source library that unifies parameter-efficient and modular transfer learning in large language models. |
| Approach: | They propose to integrate 10 different methods into a unified interface for parameter-efficient and modular transfer learning in large language models. |
| Outcome: | The proposed library is able to perform on multiple NLP tasks and is open-source. |
AdapterHub: A Framework for Adapting Transformers (2020.emnlp-demos)
Copied to clipboard
Jonas Pfeiffer, Andreas Rücklé, Clifton Poth, Aishwarya Kamath, Ivan Vulić, Sebastian Ruder, Kyunghyun Cho, Iryna Gurevych
| Challenge: | AdapterHub framework enables dynamic “stiching-in” of pre-trained adapters for different tasks and languages. |
| Approach: | They propose a framework that allows dynamic "stiching-in" of pre-trained adapters for different tasks and languages. |
| Outcome: | The proposed framework allows dynamic “stiching-in” of pre-trained adapters for different tasks and languages. |
Adaptable Adapters (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing work uses the same adapter architecture for every dataset regardless of the properties of the dataset or the amount of training data. |
| Approach: | They propose to use adaptable adapters to finetune lightweight neural network layers on top of pretrained weights. |
| Outcome: | The proposed adapters achieve on-par performances with the standard adapter architecture while using a considerably smaller number of adapter layers. |
Zero- and Few-Shot NLP with Pretrained Language Models (2022.acl-tutorials)
Copied to clipboard
| Challenge: | a tutorial aims to introduce NLP researchers to the latest techniques for learning from little-to-no data . aims at bringing interested researchers up to speed about the latest and ongoing techniques . |
| Approach: | They aim to introduce techniques for learning from little-to-no data using pretrained language models. |
| Outcome: | This tutorial aims to bring interested NLP researchers up to speed about recent techniques . it will cover methods from manual engineering, better inference algorithms to better tuning methods . |
AMAL: Meta Knowledge-Driven Few-Shot Adapter Learning (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning pre-trained language models fail to yield meaningful results in the few-shot regime. |
| Approach: | They propose a meta-learning-driven low-rank adapter pooling method for leveraging pre-trained language models even with just a few data points. |
| Outcome: | The proposed method outperforms previous few-shot learning methods on five text classification benchmark datasets. |
Monolingual Adapters for Zero-Shot Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing adapter layers are more parameter-efficient and provide better performance than bilingual ones. |
| Approach: | They propose to use monolingual adapter layers instead of bilingual ones to compose them and generalize to unseen language pairs. |
| Outcome: | The proposed adapter layer formalism achieves a median improvement of +2.77 BLEU points over a 20-language multilingual Transformer baseline trained on TED talks. |
Systematic Analysis for Pretrained Language Model Priming for Parameter-Efficient Fine-tuning (2024.naacl-srw)
Copied to clipboard
| Challenge: | Parameter-efficient (PE) methods for adapting pre-trained language models to downstream tasks are still lacking in many cases. |
| Approach: | They propose a general PE priming framework to enhance few-shot adaptation and generalization ability of PE methods. |
| Outcome: | The proposed framework reveals that the best priming strategy facilitates adaptation to target tasks. |
Know-Adapter: Towards Knowledge-Aware Parameter-Efficient Transfer Learning for Few-shot Named Entity Recognition (2024.lrec-main)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is a fundamental task in natural language processing. |
| Approach: | They propose a knowledgeable adapter to incorporate structure and semantic knowledge of knowledge graphs into PLMs for few-shot NER. |
| Outcome: | The proposed adapter improves the quality of retrieved information by adding explicit knowledge from external sources to PEFTs. |
Prompt-free and Efficient Few-shot Learning with Language Models (2022.acl-long)
Copied to clipboard
Rabeeh Karimi Mahabadi, Luke Zettlemoyer, James Henderson, Lambert Mathias, Marzieh Saeidi, Veselin Stoyanov, Majid Yazdani
| Challenge: | Existing methods for few-shot fine-tuning of pretrained language models require carefully engineered prompts and verbalizers to convert inputs into a cloze-format that the PLM can score. |
| Approach: | They propose a method for few-shot fine-tuning of pretrained language models that uses task-specific adapters instead of manually engineered prompts and verbalizers. |
| Outcome: | The proposed method outperforms existing state-of-the-art methods on a wide range of few shot NLP tasks. |
Making Pre-trained Language Models Better Few-shot Learners (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies show that the GPT-3 model can perform few-shots on language understanding tasks with a natural-language prompt and a few task demonstrations. |
| Approach: | They propose a technique for fine-tuning language models using a few examples . they propose LM-BFF, which uses prompt-based fine-uning and a pipeline for automating prompt generation . |
| Outcome: | The proposed approach outperforms standard fine-tuning procedures on a range of NLP tasks. |