Adapt in Contexts: Retrieval-Augmented Domain Adaptation via In-Context Learning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models have demonstrated their capability with few-shot inference . however, in-domain demonstrations are not always available in real scenarios . |
| Approach: | They propose unsupervised domain adaptation problem to adapt language models from source domain to target domain without any target labels. |
| Outcome: | The proposed model performs better than baseline models on Sentiment Analysis and Named Entity Recognition tasks. |
Similar Papers
UDAPTER - Efficient Domain Adaptation Using Adapters (2023.eacl-main)
Copied to clipboard
| Challenge: | Using adapters, unsupervised domain adaptation (UDA) is more parameter efficient and requires large-scale data to be effective. |
| Approach: | They propose to add small bottleneck layers to each layer of a pre-trained language model to make it more parameter efficient by adding adapters. |
| Outcome: | The proposed methods outperform unsupervised domain adaptation methods such as DANN and DSN in natural language inference and sentiment classification tasks. |
Language Models Struggle to Use Representations Learned In-Context (2026.acl-long)
Copied to clipboard
| Challenge: | a recent study shows that large language models are capable of inducing rich representations of data that are seen in-context . a novel task, adaptive world modeling, shows that even the most performant LLMs cannot reliably leverage novel semantics defined in-constitut. |
| Approach: | They propose to use in-context representations to induce rich representations of data . they also propose to probe models using a novel task to enable flexible deployment . |
| Outcome: | The proposed model can use in-context representations to complete simple downstream tasks. |
Self-Augmented In-Context Learning for Unsupervised Word Translation (2024.acl-short)
Copied to clipboard
| Challenge: | Large language models (LLMs) have strong word translation or bilingual lexicon induction (BLI) capabilities in few-shot setups, but they cannot match the performance of ‘traditional’ mapping-based approaches in the unsupervised scenario where no seed translation pairs are available. |
| Approach: | They propose a self-augmented in-context learning method that iteratively induces a set of high-confidence word translation pairs for in-constext learning from an LLM and reapplies them to the same LLM in the ICL fashion. |
| Outcome: | The proposed method shows substantial gains over zero-shot prompting of LLMs on two established benchmarks, outperforming mapping-based baselines across the board. |
Domain Confused Contrastive Learning for Unsupervised Domain Adaptation (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing studies on domain-shifting adaptations have focused on domain . |
| Approach: | They propose a self-supervised approach to unsupervised domain adduction using domain puzzles to bridge the source and target domains and retain discriminative representations after adaptation. |
| Outcome: | The proposed approach outperforms baselines and further ablation studies show that it is more stable and effective when performing other data augmentations. |
Semi-supervised Domain Adaptation for Dependency Parsing via Improved Contextualized Word Representations (2020.coling-main)
Copied to clipboard
| Challenge: | Recent advances in deep neural network models have improved parsing performance on in-domain texts . however, the problem is to improve performance on out-of-domain text data when there is only a small-scale out-domain labeled data. |
| Approach: | They propose to use adversarial learning and fine-tuning BERT to improve contextualized word representations on out-of-domain texts. |
| Outcome: | The proposed models achieve consistent improvement and fine-tune BERT processes boost parsing accuracy by a large margin. |
Topic-guided Example Selection for Domain Adaptation in LLM-based Machine Translation (2024.eacl-srw)
Copied to clipboard
| Challenge: | Current machine translation (MT) systems perform well in domains on which they were trained, but adaptation to unseen domains remains a challenge. |
| Approach: | They propose to use large language models to adapt to unseen domains by in-context example selection. |
| Outcome: | The proposed method outperforms baselines on multilingual out-of-domain tests, though it does not match performance with strong baselines for the in-language setting. |
Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent Large Language Models (LLMs) generate factually incorrect answers based on their parametric memory. |
| Approach: | They propose a retrieval-augmented large language model that can dynamically select the most suitable strategy based on query complexity. |
| Outcome: | The proposed approach improves the performance of QA systems on open-domain QA datasets. |
UDALM: Unsupervised Domain Adaptation through Language Modeling (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing techniques for unsupervised domain adaptation (UDA) are limited by domain shift, which leads to performance degradation. |
| Approach: | They propose a fine-tuning procedure that uses a mixed classification and Masked Language Model loss to adapt to the target domain distribution in a robust and sample efficient manner. |
| Outcome: | The proposed procedure can adapt to the target domain distribution in a robust and sample efficient manner. |
The Trade-offs of Domain Adaptation for Neural Language Models (2022.acl-long)
Copied to clipboard
| Challenge: | Neural Language Models (LMs) trained on large generic training sets have been shown to be effective at adapting to smaller, specific target domains for language modeling and other downstream tasks. |
| Approach: | They propose a framework for a Neural Language Models (LM) to be presented in a common framework. |
| Outcome: | The proposed framework highlights similarities and subtle differences between adaptation techniques and the framework. |
Tracing Multilingual Knowledge Acquisition Dynamics in Domain Adaptation: A Case Study of Biomedical Adaptation (2026.eacl-long)
Copied to clipboard
| Challenge: | Multilingual domain adaptation (ML-DA) enables large language models to acquire domain knowledge across languages. |
| Approach: | They propose an adaptive evaluation method that constructs multiple-choice QA datasets from the same bilingual domain corpus used for training. |
| Outcome: | The proposed method constructs multiple-choice QA datasets from the same bilingual domain corpus used for training, thereby enabling direct analysis of multilingual knowledge acquisition. |