Non-compositional Expression Generation Based on Curriculum Learning and Continual Learning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Non-compositional expressions are a classic ‘pain in the neck’ for NLP systems because of their non-composibility and limited data resources. |
| Approach: | They propose a dynamic curriculum learning framework which learns training examples from easy ones to harder ones but suffers from the forgetting problem. |
| Outcome: | The proposed framework improves on idiomatic expression generation and metaphor generation. |
Similar Papers
Non-compositional Expression Generation and its Continual Learning (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent work shows that pre-trained language models are limited in their ability to generate non-compositional expressions. |
| Approach: | They propose a mask-infilling task to examine non-compositional expressions in English . they compare large pre-trained language models and continual learning methods . |
| Outcome: | The proposed task aims to investigate the ability of pre-trained language models to generate non-compositional expressions in English and their continual learning. |
CLCL: Non-compositional Expression Detection with Contrastive Learning and Curriculum Learning (2023.acl-long)
Copied to clipboard
| Challenge: | Non-compositional expressions are a substantial challenge for natural language processing systems, necessitating more intricate processing compared to general language tasks. |
| Approach: | They propose a dynamic curriculum learning framework specifically designed to take advantage of scarce available training data for modeling non-compositionality. |
| Outcome: | The proposed framework improves on idiom usage recognition and metaphor detection tasks. |
Rehearsal-Free Modular and Compositional Continual Learning for Language Models (2024.naacl-short)
Copied to clipboard
| Challenge: | Existing methods to overcome catastrophic forgetting are rehearsal-based and parameter isolation-based. |
| Approach: | They propose a rehearsal-free framework which continuously adds new modules to language models and composes them with existing modules. |
| Outcome: | Experiments on benchmarks show that MoCL outperforms state-of-the-art and effectively facilitates knowledge transfer. |
In-sample Curriculum Learning by Sequence Completion for Natural Language Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing work on curriculum learning rely on task-specific expertise and cannot generalize to different tasks. |
| Approach: | They propose to do in-sample curriculum learning for natural language generation tasks using human-crafted rules and a numeric score for each sample based on domain expertise to rank the model. |
| Outcome: | The proposed learning strategy generalizes well to different tasks and achieves significant improvements over baselines. |
An Unsupervised Method for Learning Representations of Multi-word Expressions for Semantic Classification (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for learning multi-word expressions have language sparsity and are not supervised. |
| Approach: | They propose an unsupervised approach to learning a compositional representation function for multi-word expressions . they use a Tratz dataset to train the composition function on the word-semantic relation . |
| Outcome: | The proposed method outperforms the previous state-of-the-art method on the Tratz dataset with an F1 score of 50.4%. |
Class-Incremental Learning based on Label Generation (2023.acl-short)
Copied to clipboard
| Challenge: | Existing studies on pre-trained language models focus on task-incremental learning (TIL) but they perform poorly in a more challenging setting of class-incremental learning. |
| Approach: | They propose a method which solves CIL based on label generation by using sparse vocabulary and creates pseudo-replay samples by using label semantics. |
| Outcome: | The proposed method outperforms baseline models by a large margin in the class-incremental learning setting. |
Fine-tuned Language Models are Continual Learners (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent work on large language models relies on intuition that most tasks can be described via natural language instructions. |
| Approach: | They propose that a model should be able to keep extending its knowledge without forgetting previous skills. |
| Outcome: | The proposed model can learn 8 new diverse language generation tasks while maintaining good performance on previous tasks, spanning in total of 70 datasets. |
Leveraging Training Dynamics and Self-Training for Text Classification (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Semi-supervised learning (SSL) is a promising technique for improving deep learning models when training data is scarce. |
| Approach: | They propose a semi-supervised learning approach that leverages training dynamics of unlabeled data. |
| Outcome: | The proposed method achieves an average increase in F1 score of 3.5% over baselines in low resource settings. |
Rethinking Long Context Generation from the Continual Learning Perspective (2025.coling-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) struggle with processing long contexts due to the limited context window. |
| Approach: | They propose to combine a limited context window with a continual learning perspective to improve LLMs' efficiency in processing long contexts. |
| Outcome: | The proposed models improve the performance of Large Language Models (LLMs) by integrating learning strategies with existing approaches. |
Continual Lifelong Learning in Natural Language Processing: A Survey (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to continual learning (CL) are costly and time-consuming. |
| Approach: | They propose to examine the problem of continual learning in NLP through the lens of various NLP tasks and provide a critical review of existing methods. |
| Outcome: | The proposed methods are critical to the development of CL models and provide a critical review of existing methods and datasets. |