| Challenge: | Existing work on domain adaptation does not exploit the structure of the input text . PBLM can naturally feed structure aware text classifiers such as LSTM and CNN . |
| Approach: | They propose a model that integrates pivot-based and NN modeling in a structure aware manner. |
| Outcome: | The proposed model can naturally feed structure aware text classifiers such as LSTM and CNN. |
Similar Papers
Task Refinement Learning for Improved Accuracy and Stability of Unsupervised Domain Adaptation (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to domain adaptation (DA) require labeled data that can be found in only a handful of domains. |
| Approach: | They propose a task-refinement learning approach to solve pivot detection problems . they propose to train PBLM models with gradually increasing information exposed about each pivot . |
| Outcome: | The proposed approach achieves state-of-the-art accuracy in six domain adaptation setups for sentiment classification. |
PERL: Pivot-based Domain Adaptation for Pre-trained Deep Contextualized Embedding Models (2020.tacl-1)
Copied to clipboard
| Challenge: | PERL is a representation learning model that uses labeled data from the source domain and unlabeled data not necessarily drawn from the target domain. |
| Approach: | They propose a model that extends contextualized word embedding models with pivot-based fine-tuning to address this bottleneck. |
| Outcome: | The proposed model outperforms strong baselines across 22 sentiment classification domain adaptation setups and improves in-domain model performance. |
Deep Pivot-Based Modeling for Cross-language Cross-domain Transfer with Minimal Guidance (D18-1)
Copied to clipboard
| Challenge: | a framework for cross-domain and cross-language transfer has hardly been explored . cross-linguistic and cross language transfer methods are used for multilingual applications . |
| Approach: | They propose a framework that builds on pivot-based learning, structure-aware Deep Neural Networks and bilingual word embeddings to train a model on labeled data from one language pair. |
| Outcome: | The proposed model outperforms existing models even when trained in the lazy setup . the proposed model can be applied to nine English-German and nine English - french domain pairs without retraining . |
Pivot-based Transfer Learning for Neural Machine Translation between Non-English Languages (D19-1)
Copied to clipboard
| Challenge: | Using parallel corpora, we train a single, direct NMT model for non-English language pairs. |
| Approach: | They propose three ways to increase the relation among source, pivot, and target languages in pre-training . they use additional adapter component to smoothly connect pre-trained encoder and decoder . |
| Outcome: | The proposed methods outperform multilingual models up to +2.6% BLEU in WMT 2019 French-German and German-Czech tasks. |
Neural Unsupervised Domain Adaptation in NLP—A Survey (2020.coling-main)
Copied to clipboard
| Challenge: | Deep neural networks excel at learning from labeled data, but learning from unlabeled data remains a challenge. |
| Approach: | They review neural unsupervised domain adaptation techniques which do not require labeled target domain data. |
| Outcome: | The proposed techniques are more challenging yet widely applicable. |
Simplified Neural Unsupervised Domain Adaptation (N19-1)
Copied to clipboard
| Challenge: | Existing unsupervised domain adaptation methods use neural networks to learn representations that are trained to predict the values of subset of important features called “pivot features.” |
| Approach: | They propose to combine the representation learner and task learner to improve on existing neural domain adaptation algorithms by removing heuristically-selected "pivot features" they show competitive performance with a simpler model. |
| Outcome: | The proposed model outperforms existing models by removing heuristically-selected pivot features. |
Neural Adaptation Layers for Cross-domain Named Entity Recognition (D18-1)
Copied to clipboard
| Challenge: | Named entity recognition is a type of information extraction task whereby features can be designed based on domain-specific knowledge. |
| Approach: | They propose to use existing neural architectures to adapt to new domains without retraining . they propose to add adaptation layers to existing neural models to minimize re-training based on source data. |
| Outcome: | The proposed approach significantly outperforms state-of-the-art methods on social media domains. |
UDAPTER - Efficient Domain Adaptation Using Adapters (2023.eacl-main)
Copied to clipboard
| Challenge: | Using adapters, unsupervised domain adaptation (UDA) is more parameter efficient and requires large-scale data to be effective. |
| Approach: | They propose to add small bottleneck layers to each layer of a pre-trained language model to make it more parameter efficient by adding adapters. |
| Outcome: | The proposed methods outperform unsupervised domain adaptation methods such as DANN and DSN in natural language inference and sentiment classification tasks. |
Domain adaptation for part-of-speech tagging of noisy user-generated text (N19-1)
Copied to clipboard
| Challenge: | Existing POS taggers for canonical German text achieve good results around 97% accuracy, but when applying these trained models to out-of-domain data the performance decreases drastically. |
| Approach: | They propose a neural network that trains an out-of-domain model on a large newswire corpus and transfers those weights by using them as a prior for a model trained on the target domain. |
| Outcome: | The proposed model achieves a tagging accuracy of slightly over 90%, improving on the previous state of the art for this task. |
Recursive Neural Structural Correspondence Network for Cross-domain Aspect and Opinion Co-Extraction (P18-1)
Copied to clipboard
| Challenge: | supervised learning methods have been used for fine-grained opinion analysis but lack of labeled data hinders learning . authors develop a recursive neural network that could reduce domain shift in word level . a recent paper shows that unsupervised methods fail to adapt well across domains . |
| Approach: | They propose a supervised neural network that reduces domain shift effectively in word level . they treat these relations as invariant "pivot information" across domains to build structural correspondences . |
| Outcome: | The proposed model reduces domain shift effectively in word level through syntactic relations . it can be used to predict the relation between two adjacent words in the dependency tree . |