| Challenge: | Existing approaches to improve machine learning performance are mixed experts and domain adversarial training. |
| Approach: | They investigate the problem of unsupervised multi-source domain adaptation . they combine predictions of multiple domain experts and combine them to induce a domain agnostic representation space . |
| Outcome: | The proposed methods improve models' performance while limiting learning time. |
Similar Papers
Multi-Source Domain Adaptation with Mixture of Experts (D18-1)
Copied to clipboard
| Challenge: | Existing methods for domain adaptation from multiple sources are designed to transfer supervision from a single source domain. |
| Approach: | They propose to capture the relationship between a target example and different source domains by a point-to-set metric. |
| Outcome: | The proposed method outperforms baselines and can handle negative transfer. |
Mixture-of-Domain-Adapters: Decoupling and Injecting Domain Knowledge to Pre-trained Language Models’ Memories (2023.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models demonstrate excellent abilities to understand texts in the generic domain while struggling in a specific domain. |
| Approach: | They propose to decouple the feed-forward networks of the Transformer architecture into two parts to maintain old-domain knowledge and a mixture-of-adapters gate to inject domain-specific knowledge in parallel. |
| Outcome: | The proposed method achieves superior performance on in-domain, out-of-domain and knowledge-intensive tasks. |
Neural Unsupervised Domain Adaptation in NLP—A Survey (2020.coling-main)
Copied to clipboard
| Challenge: | Deep neural networks excel at learning from labeled data, but learning from unlabeled data remains a challenge. |
| Approach: | They review neural unsupervised domain adaptation techniques which do not require labeled target domain data. |
| Outcome: | The proposed techniques are more challenging yet widely applicable. |
Improving the Quality Trade-Off for Neural Machine Translation Multi-Domain Adaptation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Building neural machine translation systems to perform well on a specific target domain remains a challenge. |
| Approach: | They propose to train a single NMT system per language pair that performs well across multiple domains. |
| Outcome: | The proposed approach improves the Pareto frontier on this task. |
Domain Differential Adaptation for Neural Machine Translation (D19-56)
Copied to clipboard
| Challenge: | Neural networks are data hungry and domain sensitive, so it is difficult to obtain labeled data for every domain. |
| Approach: | They propose a framework for domain adaptation where we model the difference between domains instead of smoothing over them. |
| Outcome: | The proposed framework improves on domain adaptation in multiple experimental settings. |
Simplified Neural Unsupervised Domain Adaptation (N19-1)
Copied to clipboard
| Challenge: | Existing unsupervised domain adaptation methods use neural networks to learn representations that are trained to predict the values of subset of important features called “pivot features.” |
| Approach: | They propose to combine the representation learner and task learner to improve on existing neural domain adaptation algorithms by removing heuristically-selected "pivot features" they show competitive performance with a simpler model. |
| Outcome: | The proposed model outperforms existing models by removing heuristically-selected pivot features. |
Adversarial Domain Adaptation Using Artificial Titles for Abstractive Title Generation (P19-1)
Copied to clipboard
| Challenge: | Obtaining good quality labeled data can be difficult and expensive for abstractive summarization models . authors propose the use of artificial titles for unlabeled target documents . |
| Approach: | They propose to use artificial titles and sequential training to capture grammatical style of unlabeled target domains to adapt to/from news articles and Stack Exchange posts. |
| Outcome: | The proposed techniques can boost performance for unsupervised adaptation and fine-tuning with limited target data. |
Semi-supervised Domain Adaptation for Dependency Parsing (P19-1)
Copied to clipboard
| Challenge: | Currently, most studies on cross-domain parsing focus on unsupervised domain adaptation . however, unsupervised approaches make limited progress due to the intrinsic difficulty of both domain adaptation and parse. |
| Approach: | They propose a semi-supervised domain adaptation problem for Chinese dependency parsing by using newly-annotated large-scale domain-aware datasets. |
| Outcome: | The proposed method is more effective than direct corpus concatenation and multi-task learning. |
Unsupervised Energy-based Adversarial Domain Adaptation for Cross-domain Text Classification (2021.findings-acl)
Copied to clipboard
| Challenge: | Extensive experiments on multidomain sentiment classification and yes/no question-answering classification are conducted. |
| Approach: | They propose an unsupervised energy-based adversarial domain adaptation framework that maps the text sequences from both source and target domains to a feature space. |
| Outcome: | The proposed framework improves on multidomain sentiment classification and Yes/No question-answering classification. |
Factorized Transformer for Multi-Domain Neural Machine Translation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Multi-domain Neural Machine Translation (MMT) is a challenging task due to the extreme diversity of cross-domain wording and phrasing style, and the imperfections of training data distribution. |
| Approach: | They propose a factorized NMT model that divides domain-shared knowledge into domain-specific ones that are private for each constituent domain. |
| Outcome: | The proposed model achieves state-of-the-art performance and opens up new perspectives for multi-domain and open-domain applications. |