Revisit Overconfidence for OOD Detection: Reassigned Contrastive Learning with Adaptive Class-dependent Threshold (2022.naacl-main)
Copied to clipboard
Yanan Wu, Keqing He, Yuanmeng Yan, QiXiang Gao, Zhiyuan Zeng, Fujia Zheng, Lulu Zhao, Huixing Jiang, Wei Wu, Weiran Xu
| Challenge: | Existing methods for OOD detection are based on labeled in-domain data . detecting out-of-domain (OOD) or unknown intents is challenging . |
| Approach: | They propose a novel reassigned contrastive learning method to discriminate IND intents for over-confident OOD and an adaptive class-dependent local threshold mechanism to separate similar IND and OOD intents. |
| Outcome: | The proposed method is effective for both aspects of overconfidence issues. |
Similar Papers
Modeling Discriminative Representations for Out-of-Domain Detection with Supervised Contrastive Learning (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods of OOD detection only focus on whether a sample is correctly classified . lack of real OOD examples leads to poor prior knowledge about these unknown intents . |
| Approach: | They propose a supervised contrastive learning objective to minimize intra-class variance . they employ an adversarial augmentation mechanism to obtain pseudo diverse views . |
| Outcome: | The proposed method minimizes intra-class variance by pulling together in-domain intents belonging to the same class and maximizes inter-class variation by pushing apart samples from different classes. |
The Open-World Lottery Ticket Hypothesis for OOD Intent Classification (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods of Out-of-Domain intent classification lack confidence in In- and Out- of-domain intents. |
| Approach: | They propose to prune overparameterized models to provide better confidence . they extend the Lottery Ticket Hypothesis to open-world scenarios . |
| Outcome: | The proposed model can be calibrated to distinguish In- and Out-of-domain intents . the model can also improve on open-world scenarios . |
Adversarial Self-Supervised Learning for Out-of-Domain Detection (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for detecting out-of-domain (OOD) intents are unsupervised and require extensive labeled data. |
| Approach: | They propose a self-supervised contrastive learning framework to model discriminative semantic features from unlabeled data. |
| Outcome: | The proposed framework outperforms baseline methods on two public benchmark datasets with a statistically significant margin. |
Watch the Neighbors: A Unified K-Nearest Neighbor Contrastive Learning Framework for OOD Intent Discovery (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for finding out-of-domain intents suffer from in-domain overfitting problem . previous methods fail to transfer prior knowledge to downstream clustering . |
| Approach: | They propose a unified K-nearest neighbor contrastive learning framework to discover OOD intents . they propose IND pre-training objective to learn discriminative features while maintaining intra-class diversity . |
| Outcome: | The proposed framework improves on three benchmark datasets. |
Memory-Based Invariance Learning for Out-of-Domain Text Classification (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to learning invariant representations rely on the assumption that training and test sets come from the same domain. |
| Approach: | They propose to extend a classification model trained on multiple source domains to an unseen target domain by using key-value memory. |
| Outcome: | The proposed method improves on sentiment analysis and natural language inference tasks. |
Semantic Role Labeling Guided Out-of-distribution Detection (2024.lrec-main)
Copied to clipboard
Jinan Zou, Maihao Guo, Yu Tian, Yuhao Lin, Haiyao Cao, Lingqiao Liu, Ehsan Abbasnejad, Javen Qinfeng Shi
| Challenge: | Existing methods for identifying domain-shifted instances are prone to OOD and adversarial inputs. |
| Approach: | They propose an unsupervised method that separates, extracts, and learns the semantic role labeling guided out-of-distribution Detection (SRLOOD) they propose a self-supervised approach to enhance global-local feature learning by predicting SRL extracted role. |
| Outcome: | The proposed method achieves SOTA performance on four OOD benchmarks. |
Beyond the Known: Investigating LLMs Performance on Out-of-Domain Intent Detection (2024.lrec-main)
Copied to clipboard
Pei Wang, Keqing He, Yejie Wang, Xiaoshuai Song, Yutao Mou, Jingang Wang, Yunsen Xian, Xunliang Cai, Weiran Xu
| Challenge: | Out-of-domain (OOD) intent detection is crucial for task-oriented dialogue systems. |
| Approach: | They conduct a comprehensive evaluation of large language models (LLMs) under various experimental settings and outline their strengths and weaknesses. |
| Outcome: | The proposed models exhibit strong zero-shot and few-shot capabilities, but is still at a disadvantage compared to models fine-tuned with full resource. |
KNN-Contrastive Learning for Out-of-Domain Intent Classification (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for OOD intent classification are limited to regions with compact or simply-connected features, which assumes no OOD intentions reside. |
| Approach: | They propose a method that uses k-nearest neighbors to learn discriminative semantic features that are more conducive to OOD detection. |
| Outcome: | The proposed method improves OOD detection performance while requiring no restrictions on feature distribution. |
Disentangled Knowledge Transfer for OOD Intent Discovery with Unified Contrastive Learning (2022.acl-short)
Copied to clipboard
| Challenge: | Existing methods to find out out-of-domain (OOD) intents do not take prior knowledge of in-domain data into account. |
| Approach: | They propose a disentangled knowledge transfer method to bridge the gap between IND pre-training and OOD clustering by using a unified multi-head contrastive learning framework. |
| Outcome: | The proposed method is able to group new unknown intents into different clusters, enabling future development of the system. |
UniNL: Aligning Representation Learning with Scoring Function for OOD Detection via Unified Neighborhood Learning (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to detect out-of-domain (OOD) intents ignore alignment between representation learning and scoring function, limiting performance. |
| Approach: | They propose a unified neighborhood learning framework to detect OOD intents . they propose to align representation learning with scoring function . |
| Outcome: | The proposed method is able to detect out-of-domain (OOD) intents from user queries. |