What’s in a Domain? Learning Domain-Robust Text Representations using Adversarial Training (N18-2)
Copied to clipboard
| Challenge: | a key roadblock is application to new domains, unseen in training. |
| Approach: | They propose a method to optimise in- and out-of-domain accuracy by combing domain-specific and domain-general components with adversarial training for domain. |
| Outcome: | The proposed method improves on domain adaptation and domain-adversarial training. |
Similar Papers
Robust Semantic Parsing with Adversarial Learning for Domain Generalization (N19-2)
Copied to clipboard
| Challenge: | Using adversarial learning to train models on a higher level of abstraction to increase their robustness to lexical and stylistic variations is crucial for the integration of Semantic Parsing technologies in real applications. |
| Approach: | They propose to perform Semantic Parsing with a domain classification adversarial task and an unsupervised domain discovery approach that yields equivalent improvements. |
| Outcome: | The proposed approach improves on a French corpus of encyclopedic documents annotated with FrameNet and an unsupervised domain discovery approach yields equivalent improvements. |
The Trade-offs of Domain Adaptation for Neural Language Models (2022.acl-long)
Copied to clipboard
| Challenge: | Neural Language Models (LMs) trained on large generic training sets have been shown to be effective at adapting to smaller, specific target domains for language modeling and other downstream tasks. |
| Approach: | They propose a framework for a Neural Language Models (LM) to be presented in a common framework. |
| Outcome: | The proposed framework highlights similarities and subtle differences between adaptation techniques and the framework. |
End-to-End Bias Mitigation by Modelling Biases in Corpora (2020.acl-main)
Copied to clipboard
| Challenge: | Recent studies have shown that strong natural language understanding models are prone to relying on unwanted dataset biases without learning the underlying task. |
| Approach: | They propose two learning strategies to train neural models that are more robust to dataset biases and transfer better to out-of-domain datasets. |
| Outcome: | The proposed methods improve robustness in all settings and transfer better to out-of-domain datasets. |
Learning Domain Representation for Multi-Domain Sentiment Classification (N18-1)
Copied to clipboard
| Challenge: | Training data for sentiment analysis is abundant in multiple domains, yet scarce for other domains. |
| Approach: | They propose to use domain-specific representations of input sentences to improve sentiment classification . they use a descriptor vector to map adversarially trained domain-general Bi-LSTM inputs into domain- specific representations . |
| Outcome: | The proposed model outperforms existing methods on multi-domain sentiment analysis significantly. |
Domain Adversarial Fine-Tuning as an Effective Regularizer (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing fine-tuning techniques can degrade general-domain representations . however, fine-timing can lead to catastrophic forgetting of knowledge . |
| Approach: | They propose a new regularization technique that complements the task-specific loss used during fine-tuning with an adversarial objective. |
| Outcome: | Empirical results show that AFTER improves performance on various natural language understanding tasks compared to standard fine-tuning. |
In and Out-of-Domain Text Adversarial Robustness via Label Smoothing (2023.acl-short)
Copied to clipboard
| Challenge: | Existing studies show that state-of-the-art NLP models are vulnerable to adversarial attacks . label smoothing has been proven effective in a variety of applications and modalities . |
| Approach: | They propose to use label smoothing to improve adversarial robustness in pre-trained models against various popular attacks. |
| Outcome: | The proposed method significantly improves adversarial robustness in pre-trained models against various popular attacks. |
Memory-Based Invariance Learning for Out-of-Domain Text Classification (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to learning invariant representations rely on the assumption that training and test sets come from the same domain. |
| Approach: | They propose to extend a classification model trained on multiple source domains to an unseen target domain by using key-value memory. |
| Outcome: | The proposed method improves on sentiment analysis and natural language inference tasks. |
Revisiting Multi-Domain Machine Translation (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing approaches to handle multi-domain machine translation systems are lacking due to the variability of data. |
| Approach: | They propose to use domain adaptation methods to handle situations where a sample of matched sentences is available in training and where only samples of source-side sentences are available. |
| Outcome: | The proposed model is able to handle multiple domains and their expectations with respect to performance. |
Methods for Estimating and Improving Robustness of Language Models (2022.naacl-srw)
Copied to clipboard
| Challenge: | Large language models suffer from weak generalisation ability due to shallow textual relations over full semantic complexity of the problem. |
| Approach: | They propose to incorporate some of these measures into training objectives to enhance distributional robustness of LLMs. |
| Outcome: | The proposed models outperform human models on complex tasks and outperformed other models on deep networks. |
SemRoDe: Macro Adversarial Training to Learn Representations that are Robust to Word-Level Attacks (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing approaches to defend against word-level attacks have been limited. |
| Approach: | They propose a new approach called Semantic Robust Defence to enhance the robustness of language models by aligning the domains with a distance-based objective. |
| Outcome: | The proposed approach can be generalized across word embeddings, even when they share minimal overlap at both vocabulary and word-substitution levels. |