| Challenge: | Existing methods for out-of-domain (OOD) detection require huge effort to collect OOD sentences. |
| Approach: | They propose to use only in-domain (IND) sentences to build a generative adversarial network (GAN) of which the discriminator generates low scores for OOD sentences. |
| Outcome: | The proposed method is most accurate compared to existing methods on multi-domain dialog systems. |
Similar Papers
OodGAN: Generative Adversarial Network for Out-of-Domain Data Generation (2021.naacl-industry)
Copied to clipboard
| Challenge: | Existing models for OOD detection work with text, but they do not work directly with the text. |
| Approach: | They propose to use a sequential generative adversarial network (SeqGAN) based model to generate OOD data for a given domain automatically. |
| Outcome: | The proposed model outperforms state-of-the-art in OOD detection metrics for ROSTD and OSQ datasets. |
Adversarial Self-Supervised Learning for Out-of-Domain Detection (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for detecting out-of-domain (OOD) intents are unsupervised and require extensive labeled data. |
| Approach: | They propose a self-supervised contrastive learning framework to model discriminative semantic features from unlabeled data. |
| Outcome: | The proposed framework outperforms baseline methods on two public benchmark datasets with a statistically significant margin. |
Modeling Discriminative Representations for Out-of-Domain Detection with Supervised Contrastive Learning (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods of OOD detection only focus on whether a sample is correctly classified . lack of real OOD examples leads to poor prior knowledge about these unknown intents . |
| Approach: | They propose a supervised contrastive learning objective to minimize intra-class variance . they employ an adversarial augmentation mechanism to obtain pseudo diverse views . |
| Outcome: | The proposed method minimizes intra-class variance by pulling together in-domain intents belonging to the same class and maximizes inter-class variation by pushing apart samples from different classes. |
A Deep Generative Distance-Based Classifier for Out-of-Domain Detection with Mahalanobis Space (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for detecting out-of-domain (OOD) intents rely on manually labeled samples . a strong generative distance-based classifier can detect OOD samples in task-oriented dialog systems . |
| Approach: | They propose a generative distance-based classifier to detect out-of-domain (OOD) intents . they use Gaussian discriminant analysis to avoid over-confidence problems . |
| Outcome: | The proposed method outperforms baseline methods on four benchmark datasets. |
Out-of-Domain Detection for Low-Resource Text Classification Tasks (D19-1)
Copied to clipboard
| Challenge: | Existing methods for OOD detection and ID classification tasks require massive amounts of ID labeled data and no OOD labeles. |
| Approach: | They propose to use OOD-resistant Prototypical Network to detect OOD cases with limited in-domain (ID) training data to solve this task. |
| Outcome: | The proposed solution outperforms state-of-the-art methods in zero-shot OOD detection task while maintaining a competitive performance on ID classification task. |
Beyond the Known: Investigating LLMs Performance on Out-of-Domain Intent Detection (2024.lrec-main)
Copied to clipboard
Pei Wang, Keqing He, Yejie Wang, Xiaoshuai Song, Yutao Mou, Jingang Wang, Yunsen Xian, Xunliang Cai, Weiran Xu
| Challenge: | Out-of-domain (OOD) intent detection is crucial for task-oriented dialogue systems. |
| Approach: | They conduct a comprehensive evaluation of large language models (LLMs) under various experimental settings and outline their strengths and weaknesses. |
| Outcome: | The proposed models exhibit strong zero-shot and few-shot capabilities, but is still at a disadvantage compared to models fine-tuned with full resource. |
OutFlip: Generating Examples for Unknown Intent Detection with Natural Language Attack (2021.findings-acl)
Copied to clipboard
| Challenge: | Out-of-domain (OOD) input detection is vital in task-oriented dialogue systems . accepted OOD inputs lead to incorrect response of the system . |
| Approach: | They propose a method to generate out-of-domain samples from in-domain training datasets using OutFlip. |
| Outcome: | The proposed method significantly improves an intent classification model's out-of-domain detection performance. |
Revisit Overconfidence for OOD Detection: Reassigned Contrastive Learning with Adaptive Class-dependent Threshold (2022.naacl-main)
Copied to clipboard
Yanan Wu, Keqing He, Yuanmeng Yan, QiXiang Gao, Zhiyuan Zeng, Fujia Zheng, Lulu Zhao, Huixing Jiang, Wei Wu, Weiran Xu
| Challenge: | Existing methods for OOD detection are based on labeled in-domain data . detecting out-of-domain (OOD) or unknown intents is challenging . |
| Approach: | They propose a novel reassigned contrastive learning method to discriminate IND intents for over-confident OOD and an adaptive class-dependent local threshold mechanism to separate similar IND and OOD intents. |
| Outcome: | The proposed method is effective for both aspects of overconfidence issues. |
Improving Unsupervised Out-of-domain detection through Pseudo Labeling and Learning (2023.findings-eacl)
Copied to clipboard
| Challenge: | Unsupervised OOD detection is a task aimed at discriminating whether given samples are from the in-domain (IND) . previous studies adopted the one-class classification approach, assuming that the training samples come from a single domain. |
| Approach: | They propose a framework that leverages latent categorical information to improve representation learning for textual OOD detection. |
| Outcome: | The proposed framework significantly outperforms baseline models on three datasets. |
VI-OOD: A Unified Framework of Representation Learning for Textual Out-of-distribution Detection (2024.lrec-main)
Copied to clipboard
| Challenge: | Out-of-distribution (OOD) detection is a crucial part of deep neural networks. |
| Approach: | They propose a variational inference framework which maximizes the likelihood of the joint distribution p(x, y) instead of p[y|x). |
| Outcome: | The proposed framework maximizes the likelihood of the joint distribution p(x, y) instead of p[y|x). |