| Challenge: | Domain classification is the task to map spoken language utterances to one of the natural language understanding domains in intelligent personal digital assistants. |
| Approach: | They propose a neural-based approach for continuous domain adaption with normalization and regularization to accommodate new domains. |
| Outcome: | The proposed approach outperforms baseline methods on accommodated new domains and existing known domains by a large margin. |
Similar Papers
Efficient Large-Scale Neural Domain Classification with Personalized Attention (P18-1)
Copied to clipboard
| Challenge: | Using a scalable neural model, we show that personalization improves domain classification accuracy in a setting with thousands of overlapping domains. |
| Approach: | They propose a scalable neural model architecture with a shared encoder that incorporates personalization information and domain-specific classifiers that solves the problem efficiently. |
| Outcome: | The proposed architecture achieves two orders of magnitude faster than full model retraining. |
Unsupervised Domain Adaptation for Text Classification via Meta Self-Paced Learning (2022.coling-1)
Copied to clipboard
| Challenge: | Recent methods addressing unsupervised domain adaptation for textual tasks extracted domain-invariant representations through balancing between multiple objectives to align feature spaces between source and target domains. |
| Approach: | They propose to use meta-learning framework to train a neural network-based self-paced learning procedure in an end-to-end manner. |
| Outcome: | The proposed method significantly improves performance on target domains, surpassing state-of-the-art approaches. |
Neural Unsupervised Domain Adaptation in NLP—A Survey (2020.coling-main)
Copied to clipboard
| Challenge: | Deep neural networks excel at learning from labeled data, but learning from unlabeled data remains a challenge. |
| Approach: | They review neural unsupervised domain adaptation techniques which do not require labeled target domain data. |
| Outcome: | The proposed techniques are more challenging yet widely applicable. |
Hyperparameter-free Continuous Learning for Domain Classification in Natural Language Understanding (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing continual learning approaches suffer from low accuracy and performance fluctuation when the distributions of old and new data are significantly different. |
| Approach: | They propose a hyperparameter-free continual learning model for text data that can stably produce high performance under various environments. |
| Outcome: | The proposed model outperforms the best state-of-the-art method by 20% in average accuracy and each component contributes effectively to overall performance. |
PERL: Pivot-based Domain Adaptation for Pre-trained Deep Contextualized Embedding Models (2020.tacl-1)
Copied to clipboard
| Challenge: | PERL is a representation learning model that uses labeled data from the source domain and unlabeled data not necessarily drawn from the target domain. |
| Approach: | They propose a model that extends contextualized word embedding models with pivot-based fine-tuning to address this bottleneck. |
| Outcome: | The proposed model outperforms strong baselines across 22 sentiment classification domain adaptation setups and improves in-domain model performance. |
Feature Adaptation of Pre-Trained Language Models across Languages and Domains with Robust Self-Training (2020.emnlp-main)
Copied to clipboard
| Challenge: | Adapting pre-trained language models (PrLMs) to new domains has gained much attention . Adaptation of PrLMs to newdomains is important, but requires fine-tuning . |
| Approach: | They propose to use PrLMs to adapt to new domains without fine-tuning . they use class-aware feature self-distillation to learn discriminative features . |
| Outcome: | The proposed model can learn discriminative features from pre-trained language models without fine-tuning. |
Domain-Agnostic Neural Architecture for Class Incremental Continual Learning in Document Processing Platform (2023.acl-industry)
Copied to clipboard
| Challenge: | Recent methods with stochastic gradient learning struggle in streaming data setups and are restricted to specific domains. |
| Approach: | They propose a fully differentiable architecture that enables the training of high-performance classifiers when examples from each class are presented separately. |
| Outcome: | The proposed architecture achieves SOTA results without a memory buffer and clearly outperforms the reference methods. |
Continual Lifelong Learning in Natural Language Processing: A Survey (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to continual learning (CL) are costly and time-consuming. |
| Approach: | They propose to examine the problem of continual learning in NLP through the lens of various NLP tasks and provide a critical review of existing methods. |
| Outcome: | The proposed methods are critical to the development of CL models and provide a critical review of existing methods and datasets. |
Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora (2022.naacl-main)
Copied to clipboard
| Challenge: | Pretrained language models are typically learned over a large, static corpus and fine-tuned for various downstream tasks. |
| Approach: | They propose to continuously update a pretrained language model to adapt to emerging data and to keep track of the model's performance. |
| Outcome: | The proposed model can adapt to new corpora while retaining knowledge in earlier domains. |
Boosting Large Language Models with Continual Learning for Aspect-based Sentiment Analysis (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies focus on improving the performance of domain-specific models based on the target dataset. |
| Approach: | They propose a Large Language Model-based Continual Learning (LLM-CL) model for ABSA that learns the target domain’s ability while maintaining the history domains’ abilities. |
| Outcome: | The proposed model obtains new state-of-the-art over 19 datasets. |