Continual Learning with Semi-supervised Contrastive Distillation for Incremental Neural Machine Translation (2024.acl-long)
Copied to clipboard
| Challenge: | Multi-domain learning is a good solution for solving domain tasks but it requires retraining when adding a new domain. |
| Approach: | They propose to exploit unlabeled data from the same distributions of the older domains to avoid catastrophic forgetting. |
| Outcome: | The proposed framework exploits unlabeled data from the same distributions of the older domains to avoid catastrophic forgetting. |
Similar Papers
Continual Knowledge Distillation for Neural Machine Translation (2023.acl-long)
Copied to clipboard
| Challenge: | Current parallel corpora are not publicly accessible but trained models are more readily available. |
| Approach: | They propose a method to take advantage of existing translation models to improve one model of interest. |
| Outcome: | The proposed method improves on Chinese-English and German-English datasets and is robust to malicious models. |
ICL: Iterative Continual Learning for Multi-domain Neural Machine Translation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have focused on learning domain knowledge from multiple domains, but task-specific parameters hinder mutual transfer of knowledge between new domains. |
| Approach: | They propose an iterative Continual learning framework for multi-domain neural machine translation that leverages previously acquired domain knowledge. |
| Outcome: | The proposed model outperforms baseline models on UM-Corpus and OPUS datasets. |
Distill and Replay for Continual Language Learning (2020.coling-main)
Copied to clipboard
| Challenge: | Existing models fail to isolate acquired knowledge and forget previously learned tasks when learning in a stream where data distribution may shift. |
| Approach: | They propose a framework that distills knowledge and replays experience from previous tasks when fitting on a new task. |
| Outcome: | The proposed framework outperforms state-of-the-art models in continuously learning tasks of the same type but from different domains, as well as tasks of different types. |
Overcoming Catastrophic Forgetting beyond Continual Learning: Balanced Training for Neural Machine Translation (2022.acl-long)
Copied to clipboard
| Challenge: | Neural networks tend to gradually forget the previously learned knowledge when learning multiple tasks sequentially from dynamic data distributions. |
| Approach: | They propose a method that iteratively provides complementary knowledge to student models by dynamically updating teacher models trained on specific data orders. |
| Outcome: | The proposed method improves on multiple machine translation tasks and improves performance over baseline systems. |
CLASSIC: Continual and Contrastive Learning of Aspect Sentiment Classification Tasks (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have focused on continual learning of aspect sentiment classification (ASC) tasks in domain incremental learning (DIL) |
| Approach: | They propose a continual learning method that learns a sequence of tasks incrementally . they propose CLASSIC, which uses a domain incremental learning setting . |
| Outcome: | The proposed model is highly effective in a domain incremental learning setting. |
Continual Learning for Neural Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Neural machine translation models are data-driven and require large-scale training corpus . continual learning remains a big challenge for artificial intelligence systems and models . |
| Approach: | They propose a continual learning framework for NMT models that incorporates multiple stages of training to alleviate catastrophic forgetting problem. |
| Outcome: | The proposed framework achieves superior performance compared to baseline models in all settings. |
Multilingual Continual Learning using Attention Distillation (2025.coling-industry)
Copied to clipboard
| Challenge: | Existing models for Query-product relevance classification are not accurate across multiple languages. |
| Approach: | They propose a multilingual continual learning framework that adds adapters for each new language and incorporates a fusion layer above language-specific adapters. |
| Outcome: | The proposed approach reduces trainable parameters by 80% while outperforming SOTA CL methods on proprietary and external datasets. |
Serial Contrastive Knowledge Distillation for Continual Few-shot Relation Extraction (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing models for few-shot relation extraction (RE) are not suitable for continual few-sshot RE. |
| Approach: | They propose a new model to train a model for new relations with few labeled training data. |
| Outcome: | The proposed model outperforms state-of-the-art models on two benchmark datasets. |
Continual Learning of Neural Machine Translation within Low Forgetting Risk Regions (2022.emnlp-main)
Copied to clipboard
| Challenge: | Currently, continuous learning methods suffer from catastrophic forgetting problem, causing model to forget previous knowledge while learning new knowledge. |
| Approach: | They propose a two-stage continuous learning method based on local features of the real loss to avoid catastrophic forgetting problem. |
| Outcome: | The proposed method achieves significant improvements on domain adaptation and more challenging language adaptation tasks. |
Distilling Multiple Domains for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Neural machine translation is a powerful tool for high-resource domains, but performance suffers when the input domain is low-resourced. |
| Approach: | They propose a framework for training a single multi-domain neural machine translation model that can translate multiple domains without increasing inference time or memory usage. |
| Outcome: | The proposed model improves translation on both high- and low-resource domains over strong multi-domain baselines and is robust under noisy data conditions. |