Knowledge Prompting in Pre-trained Language Model for Natural Language Understanding (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge-enhanced pre-trained language models (PLMs) introduce redundant factual knowledge from knowledge bases and require complex modules. |
| Approach: | They propose a knowledge prompting-based PLM framework that incorporates factual knowledge into PLMs. |
| Outcome: | The proposed framework can be flexibly combined with existing mainstream PLMs. |
Similar Papers
Iteratively Prompt Pre-trained Language Models for Chain of Thought (2022.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) internalize a great amount of knowledge, but have been shown incapable of recalling this knowledge to solve complex & multi-step reasoning tasks. |
| Approach: | They propose an iterative prompting framework which progressively elicits relevant knowledge from PLMs for multi-step inference. |
| Outcome: | The proposed prompting framework outperforms existing prompting methods on three datasets involving multi-step reasoning. |
Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification (2022.acl-long)
Copied to clipboard
| Challenge: | Recent studies suggest that pre-trained language models have gained rich knowledge during pre-training. |
| Approach: | They propose to tune pre-trained language models with task-specific prompts to improve and stabilize prompttuning. |
| Outcome: | Extensive experiments on zero and few-shot text classification tasks show that prompt-tuning improves and stabilizes prompttun-ing. |
Give Me the Facts! A Survey on Factual Knowledge Probing in Pre-trained Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained language models are trained on vast unlabeled data, rich in world knowledge. |
| Approach: | They propose a categorization scheme for factual probing methods based on how inputs, outputs and probed PLMs are adapted . they synthesize insights about knowledge retention and prompt optimization in PLM models and analyze obstacles to adopting them as knowledge bases . |
| Outcome: | The proposed method synthesizes insights about knowledge retention and prompt optimization in PLMs, analyzes obstacles to adopting them as knowledge bases and outline directions for future work. |
Prompting Language Models for Linguistic Structure (2023.acl-long)
Copied to clipboard
| Challenge: | Existing prompting methods can test this hypothesis on autoregressive PLMs. |
| Approach: | They propose a structured prompting approach for linguistic structured prediction tasks that performs zero- and few-shot sequence tagging with autoregressive PLMs. |
| Outcome: | The proposed approach shows that the model can perform few-shot sequence tagging on part-of-speech taging, named entity recognition, and sentence chunking tasks. |
Pre-trained Language Models Can be Fully Zero-Shot Learners (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to pre-trained language models require fine-tuning on labeled datasets or manually constructing proper prompts. |
| Approach: | They propose a nonparametric prompting PLM for fully zero-shot language understanding . they compare it to previous methods for text classification and text entailment . |
| Outcome: | The proposed method outperforms previous methods on diverse tasks. |
Prompt2Model: Generating Deployable Models from Natural Language Instructions (2023.emnlp-demo)
Copied to clipboard
| Challenge: | Large language models (LLMs) are a step backward from traditional special-purpose NLP models . they require extensive computational resources for deployment and can be gated behind APIs . |
| Approach: | They propose a general-purpose method that takes a natural language task description and uses it to train a special-purpose model. |
| Outcome: | The proposed method outperforms a strong LLM by 20% while being 700 times smaller. |
LLM as Prompter: Low-resource Inductive Reasoning on Arbitrary Knowledge Graphs (2024.findings-acl)
Copied to clipboard
| Challenge: | Knowledge Graph (KG) inductive reasoning is widely adopted in various applications. |
| Approach: | They propose a framework for low-resource inductive reasoning using Large Language Models to generate a graph-structural prompt for pre-trained KGs. |
| Outcome: | The proposed framework outperforms previous methods in three-shot, one-shot and zero-shot reasoning tasks. |
AdaPrompt: Adaptive Model Training for Prompt-based NLP (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Prompt-based learning can tackle zero-shot and few-shot NLP tasks . authors propose a method that makes use of pre-trained language models . |
| Approach: | They propose to map NLP tasks into natural language prompts, which are then filled by pre-trained language models. |
| Outcome: | The proposed method outperforms standard prompt-based methods in few-shot settings. |
Adversarial Knowledge Stimulated Contrastive Prompting for Few-shot Language Learners (2023.findings-acl)
Copied to clipboard
| Challenge: | Prompt-based fine-tuning has boosted performance of Pre-trained language models on few-shot Natural Language Understanding (NLU) tasks by employing task-specific prompts. |
| Approach: | They propose a Cloze-driven prompt framework for prompt tuning that implicitly stimulates knowledge from pre-trained language models. |
| Outcome: | The proposed framework outperforms state-of-the-art for prompt-based fine-tuning on few-shot NLU tasks. |
PromptExplainer: Explaining Language Models through Prompt-based Learning (2024.findings-eacl)
Copied to clipboard
| Challenge: | Existing explanation methods rely on linear approximations, accentuating irrelevant input tokens. |
| Approach: | They propose a method that aligns the explanation process with the masked language modeling task of pretrained language models and leverages prompt-based learning to generate class-dependent explanations. |
| Outcome: | Extensive experiments show that PromptExplainer outperforms state-of-the-art explanation methods. |