Prompt Conditioned VAE: Enhancing Generative Replay for Lifelong Learning in Task-Oriented Dialogue (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing generative replay methods use only a single task-specific token to control their models. |
| Approach: | They propose a method to capture task-specific distributions with a conditional variational autoencoder, conditioned on natural language prompts to guide the pseudo-sample generation. |
| Outcome: | The proposed method outperforms baselines on natural language understanding tasks of advanced task-oriented dialogue (ToD) systems. |
Similar Papers
Prototype Conditioned Generative Replay for Continual Learning in NLP (2025.naacl-long)
Copied to clipboard
| Challenge: | Generative replay methods that rely on a single task-specific token or prompt often fail to generate pseudo-samples that accurately reflect the true data distribution. |
| Approach: | They propose a Prototype Conditioned Generative Replay method which incorporates task-level statistics into a prototyping process. |
| Outcome: | The proposed method outperforms state-of-the-art (SOTA) methods on two different scenarios. |
Prompt Augmented Generative Replay via Supervised Contrastive Learning for Lifelong Intent Detection (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to identify all possible user intents at design time are expensive and require storage of past data. |
| Approach: | They propose to continually train an intent detector on new intents while maintaining performance on prior intents. |
| Outcome: | The proposed method outperforms exemplar replay-based approaches on lifelong intent detection tasks and achieves state-of-the-art on four public datasets. |
Prompts Can Play Lottery Tickets Well: Achieving Lifelong Information Extraction via Lottery Prompt Tuning (2023.acl-long)
Copied to clipboard
| Challenge: | Existing research on information extraction tasks focuses on one specific task, but in real-world scenarios, new data of different IE tasks and domains come in a stream over time. |
| Approach: | They propose a parameter- and deployment-efficient prompt tuning method to evaluate the UIE system under a “lifelong learning” setting. |
| Outcome: | The proposed method is able to learn new tasks without forgetting old ones and expand knowledge and functionalities without retraining the whole system. |
PromptGen: Automatically Generate Prompts using Generative Models (2022.findings-naacl)
Copied to clipboard
| Challenge: | Recent prompt learning has received significant attention, where downstream tasks are reformulated to the mask-filling task with the help of a textual prompt. |
| Approach: | They propose a model PromptGen which can automatically generate prompts conditional on the input sentence. |
| Outcome: | The proposed model outperforms baseline models on the knowledge probing LAMA benchmark. |
Generative Replay Inspired by Hippocampal Memory Indexing for Continual Language Learning (2023.eacl-main)
Copied to clipboard
| Challenge: | Continual learning (CL) is a fundamental requirement for human-like general intelligence (Parisi et al., 2019). |
| Approach: | They propose to control sample generation using compressed features of previous training samples by using hippocampal memory indexing to enhance the generative replay. |
| Outcome: | The proposed method outperforms current generative replay methods and generates training samples from previous tasks. |
An Adaptive Prompt Generation Framework for Task-oriented Dialogue System (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing black-box large language models (LLMs) have excellent performance in task-oriented dialogue (TOD) tasks, but obtaining suitable prompts for specific tasks is challenging. |
| Approach: | They propose a black-box large language model that generates domain and slot information in the belief state, which serves as prior knowledge for subsequent prompt generation. |
| Outcome: | The proposed framework outperforms existing prompting methods on the MultiWOZ 2.0 dataset. |
CAP: Controllable Alignment Prompting for Unlearning in LLMs (2026.acl-long)
Copied to clipboard
Zhaokun Wang, Jinyu Guo, Jingwen Pu, Hongli Pu, Meng Yang, Xunlei Chen, Jie Ou, Wenyi Li, Guangchun Luo, Wenhong Tian
| Challenge: | Existing methods for modifying parameters are unsystematic and rely on empirical experience. |
| Approach: | They propose a controllable alignment prompting for unlearning framework that decouples unlearning into a learnable prompt optimization process via reinforcement learning. |
| Outcome: | The proposed framework achieves precise, controllable unlearning without updating model parameters. |
TaCIE: Enhancing Instruction Comprehension in Large Language Models through Task-Centred Instruction Evolution (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning Large Language Models (LLMs) encounter performance limitations, impeding further enhancements in code generation tasks. |
| Approach: | They propose to combine two distinct prompts through a hybridization process to enhance the evolution of training prompts for code LLMs. |
| Outcome: | The proposed method significantly improves the performance of Code LLMs across five code generation benchmarks. |
PromptWizard: Optimizing Prompts via Task-Aware, Feedback-Driven Self-Evolution (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have transformed AI across diverse domains, with prompting being central to their success in guiding model outputs. |
| Approach: | They propose a framework for discrete prompt optimization that generates human-readable prompts using feedback-driven critique and synthesis process. |
| Outcome: | The proposed framework improves prompt quality across 45 tasks and reduces API calls, token usage and overall cost. |
Q-Tuning: Queue-based Prompt Tuning for Lifelong Few-shot Language Learning (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for continual prompt tuning are limited by the ever-growing parameter scale of modern language models (e.g., GPT-4 that may have 1.76 trillion parameters). |
| Approach: | They propose a method for continual prompt tuning that enables the lifelong learning of a pre-trained language model by adding a task-specific prompt to a queue of older tasks. |
| Outcome: | The proposed method outperforms the state-of-the-art methods substantially on continual prompt tuning benchmarks. |