Evaluation of Transfer Learning and Domain Adaptation for Analyzing German-Speaking Job Advertisements (2022.lrec-1)
Copied to clipboard
| Challenge: | a paper presents text mining approaches on German-speaking job advertisements . transfer learning and domain adaptation are used to build text mining applications . |
| Approach: | They propose text mining approaches on German-speaking job advertisements . they use transfer learning and domain adaptation to build language models adapted to job ads . |
| Outcome: | The proposed approaches outperform general-domain language models pre-trained on ten times more data. |
Similar Papers
On the Impact of Cross-Domain Data on German Language Models (2023.findings-emnlp)
Copied to clipboard
Amin Dada, Aokun Chen, Cheng Peng, Kaleb Smith, Ahmad Idrissi-Yaghir, Constantin Seibold, Jianning Li, Lars Heiliger, Christoph Friedrich, Daniel Truhn, Jan Egger, Jiang Bian, Jens Kleesiek, Yonghui Wu
| Challenge: | Traditionally, large language models have been trained on general web crawls or domain-specific data. |
| Approach: | They present a German dataset and a dataset aimed at containing high-quality data to examine the importance of data diversity over quality. |
| Outcome: | The proposed model outperforms models trained on quality data on multiple downstream tasks. |
Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks (2020.acl-main)
Copied to clipboard
Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, Noah A. Smith
| Challenge: | Language models prerained on text from a wide variety of sources form the foundation of today’s NLP. |
| Approach: | They propose to tailor a pretrained model to the domain of a target task by using domain-adaptive pretraining in-domain. |
| Outcome: | The proposed model can be tailored to the domain of a target task and perform well under both high- and low-resource settings. |
Neural Unsupervised Domain Adaptation in NLP—A Survey (2020.coling-main)
Copied to clipboard
| Challenge: | Deep neural networks excel at learning from labeled data, but learning from unlabeled data remains a challenge. |
| Approach: | They review neural unsupervised domain adaptation techniques which do not require labeled target domain data. |
| Outcome: | The proposed techniques are more challenging yet widely applicable. |
The Trade-offs of Domain Adaptation for Neural Language Models (2022.acl-long)
Copied to clipboard
| Challenge: | Neural Language Models (LMs) trained on large generic training sets have been shown to be effective at adapting to smaller, specific target domains for language modeling and other downstream tasks. |
| Approach: | They propose a framework for a Neural Language Models (LM) to be presented in a common framework. |
| Outcome: | The proposed framework highlights similarities and subtle differences between adaptation techniques and the framework. |
Learning Strategies for Robust Argument Mining: An Analysis of Variations in Language and Domain (2024.lrec-main)
Copied to clipboard
| Challenge: | Argument mining is a complex process that requires a large amount of resources and time. |
| Approach: | They propose to analyze arguments in three different languages and domains to understand their robustness to natural language variations. |
| Outcome: | The proposed systems are more robust to natural language variations than existing arguments mining systems. |
Language adaptation experiments via cross-lingual embeddings for related languages (L18-1)
Copied to clipboard
| Challenge: | Language Adaptation is a general approach to extend existing resources from a better resourced language to a lesser resourced one. |
| Approach: | They propose to exploit lexical and grammatical similarity between languages when they are related by using orthographic similarity. |
| Outcome: | The proposed method improves the state of the art in induction of bilingual lexicons . it also improves induction performance in the Named-Entity Recognition task . |
Evaluating Domain Adaptation for Machine Translation Across Scenarios (L18-1)
Copied to clipboard
| Challenge: | Statistical machine translation (SMT) has been the dominant approach for the last 20 years, with neural machine translation becoming the new main paradigm in academic research and the industry. |
| Approach: | They propose to compare domain-adapted statistical and neural machine translation systems on three different domains and language pairs with varying degrees of domain specificity and available training data. |
| Outcome: | The proposed system is the best choice for translation, with marked impacts for domains with higher specificity. |
Domain Adaptive Inference for Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Neural Machine Translation models are effective when trained on broad domains with large datasets, such as news translation. |
| Approach: | They propose a novel approach for adaptive ensemble weighting for Neural Machine Translation by extending Bayesian Interpolation with source information. |
| Outcome: | The proposed approach improves performance on Spanish-English and English-German tasks without the need for the domain label. |
Revisiting Multi-Domain Machine Translation (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing approaches to handle multi-domain machine translation systems are lacking due to the variability of data. |
| Approach: | They propose to use domain adaptation methods to handle situations where a sample of matched sentences is available in training and where only samples of source-side sentences are available. |
| Outcome: | The proposed model is able to handle multiple domains and their expectations with respect to performance. |
Adapt in Contexts: Retrieval-Augmented Domain Adaptation via In-Context Learning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models have demonstrated their capability with few-shot inference . however, in-domain demonstrations are not always available in real scenarios . |
| Approach: | They propose unsupervised domain adaptation problem to adapt language models from source domain to target domain without any target labels. |
| Outcome: | The proposed model performs better than baseline models on Sentiment Analysis and Named Entity Recognition tasks. |