Learn Continually, Generalize Rapidly: Lifelong Knowledge Accumulation for Few-shot Learning (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models that pursue rapid generalization to new tasks are mostly trained in a single shot on fixed datasets, unable to dynamically expand their knowledge. |
| Approach: | They propose a new learning setup that assumes a model learns from a sequence of diverse NLP tasks arriving sequentially, accumulating knowledge for improved generalization to new tasks. |
| Outcome: | The proposed learning setup improves generalization ability while retaining performance on the tasks learned earlier. |
Similar Papers
Continual Training of Language Models for Few-Shot Learning (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent work on applying large language models (LMs) achieves impressive performance in many NLP applications. |
| Approach: | They propose to continuously post-train an LM with unlabeled domains to expand its knowledge without forgetting previous skills. |
| Outcome: | The proposed system improves few-shot end-task learning in these domains. |
Preserving Generalization of Language models in Few-shot Continual Relation Extraction (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for Few-shot Continual Relations Extraction (FCRE) are limited in labeled training data and models must learn from a few new samples to solve new tasks. |
| Approach: | They propose a method that leverages often-discarded language model heads to integrate knowledge from new relations with limited labeled data while avoiding catastrophic forgetting. |
| Outcome: | The proposed method circumvents catastrophic forgetting and preserves prior knowledge from pre-trained backbones while maintaining accuracy of existing classifications. |
Continual Few-shot Relation Learning via Embedding Space Regularization and Data Augmentation (2022.acl-long)
Copied to clipboard
| Challenge: | Existing continual relation learning methods rely on labeled training data for learning new tasks, which can be expensive and time-consuming. |
| Approach: | They propose a method that embeds space regularization and data augmentation to learn relational patterns with very few labeled data while avoiding catastrophic forgetting of previous task knowledge. |
| Outcome: | The proposed method outperforms existing state-of-the-art methods in CFRL task settings. |
Continual Few-Shot Learning for Text Classification (2021.emnlp-main)
Copied to clipboard
| Challenge: | a large number of end-to-end systems are needed for many tasks in natural language processing. |
| Approach: | They propose a continual few-shot learning task where a system is asked to correct mistakes with a few training examples. |
| Outcome: | The proposed task compares two NLI and one sentiment analysis datasets with baselines from diverse paradigms. |
X-Shot: A Unified System to Handle Frequent, Few-shot and Zero-shot Learning Simultaneously in Classification (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have focused on few-shot and zero-shot learning, but label occurrences vary widely . authors propose a new classification challenge that can be used to manage labels across the full frequency spectrum . |
| Approach: | They propose a new classification challenge that allows for label co-occurrences without predefined limits. |
| Outcome: | The proposed system can handle freq-shot, few-shot and zero-shot labels without limits. |
Fine-tuned Language Models are Continual Learners (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent work on large language models relies on intuition that most tasks can be described via natural language instructions. |
| Approach: | They propose that a model should be able to keep extending its knowledge without forgetting previous skills. |
| Outcome: | The proposed model can learn 8 new diverse language generation tasks while maintaining good performance on previous tasks, spanning in total of 70 datasets. |
Large-scale Lifelong Learning of In-context Instructions and How to Tackle It (2023.acl-long)
Copied to clipboard
| Challenge: | In-context instruction learning is a method to improve the target PLM’s instance- and task-level generalization performance as it observes more tasks. |
| Approach: | They propose to fine-tune a Pre-trained Language Model (PLM) on a set of tasks with in-context instructions and to extend this property to a scenario in which tasks are fed to the target PLM in a sequential manner. |
| Outcome: | The proposed method achieves noticeable improvements in both types of generalization, nearly reaching the upper bound performance obtained through joint training. |
Revisit Few-shot Intent Classification with PLMs: Direct Fine-tuning vs. Continual Pre-training (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent progress in intent detection relies on deep models and datasets with well-crafted annotations. |
| Approach: | They propose a continual pre-training approach to train deep learning models . they propose augmentation method and sequential self-distillation to boost performance . |
| Outcome: | The proposed method outperforms methods that employ continual pre-training on labeled datasets on few-shot intent detection tasks. |
Mutual-pairing Data Augmentation for Fewshot Continual Relation Extraction (2025.naacl-long)
Copied to clipboard
Nguyen Hoang Anh, Quyen Tran, Thanh Xuan Nguyen, Nguyen Thi Ngoc Diep, Linh Ngo Van, Thien Huu Nguyen, Trung Le
| Challenge: | Existing methods for Few-shot Continual Relation Extraction struggle with catastrophic forgetting and overfitting. |
| Approach: | They propose a method that transforms single input sentences into complex texts by integrating old and new data. |
| Outcome: | The proposed method sharpens model focus and improves model performance . it also uncovers fascinating behaviors of Sharpness-Aware Minimization (SAM) in Few-shot Continual Learning. |
Continual Learning for Natural Language Generation in Task-oriented Dialog Systems (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing neural approaches for natural language generation are typically developed offline for specific domains. |
| Approach: | They propose a method to expand NLG knowledge incrementally to new domains . major challenge is catastrophic forgetting, meaning a model forgets the knowledge it has learned before . |
| Outcome: | The proposed method outperforms other methods by effectively mitigating catastrophic forgetting issue. |