UniNL: Aligning Representation Learning with Scoring Function for OOD Detection via Unified Neighborhood Learning (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to detect out-of-domain (OOD) intents ignore alignment between representation learning and scoring function, limiting performance. |
| Approach: | They propose a unified neighborhood learning framework to detect OOD intents . they propose to align representation learning with scoring function . |
| Outcome: | The proposed method is able to detect out-of-domain (OOD) intents from user queries. |
Similar Papers
Watch the Neighbors: A Unified K-Nearest Neighbor Contrastive Learning Framework for OOD Intent Discovery (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for finding out-of-domain intents suffer from in-domain overfitting problem . previous methods fail to transfer prior knowledge to downstream clustering . |
| Approach: | They propose a unified K-nearest neighbor contrastive learning framework to discover OOD intents . they propose IND pre-training objective to learn discriminative features while maintaining intra-class diversity . |
| Outcome: | The proposed framework improves on three benchmark datasets. |
KNN-Contrastive Learning for Out-of-Domain Intent Classification (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for OOD intent classification are limited to regions with compact or simply-connected features, which assumes no OOD intentions reside. |
| Approach: | They propose a method that uses k-nearest neighbors to learn discriminative semantic features that are more conducive to OOD detection. |
| Outcome: | The proposed method improves OOD detection performance while requiring no restrictions on feature distribution. |
Adversarial Self-Supervised Learning for Out-of-Domain Detection (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for detecting out-of-domain (OOD) intents are unsupervised and require extensive labeled data. |
| Approach: | They propose a self-supervised contrastive learning framework to model discriminative semantic features from unlabeled data. |
| Outcome: | The proposed framework outperforms baseline methods on two public benchmark datasets with a statistically significant margin. |
Beyond the Known: Investigating LLMs Performance on Out-of-Domain Intent Detection (2024.lrec-main)
Copied to clipboard
Pei Wang, Keqing He, Yejie Wang, Xiaoshuai Song, Yutao Mou, Jingang Wang, Yunsen Xian, Xunliang Cai, Weiran Xu
| Challenge: | Out-of-domain (OOD) intent detection is crucial for task-oriented dialogue systems. |
| Approach: | They conduct a comprehensive evaluation of large language models (LLMs) under various experimental settings and outline their strengths and weaknesses. |
| Outcome: | The proposed models exhibit strong zero-shot and few-shot capabilities, but is still at a disadvantage compared to models fine-tuned with full resource. |
Modeling Discriminative Representations for Out-of-Domain Detection with Supervised Contrastive Learning (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods of OOD detection only focus on whether a sample is correctly classified . lack of real OOD examples leads to poor prior knowledge about these unknown intents . |
| Approach: | They propose a supervised contrastive learning objective to minimize intra-class variance . they employ an adversarial augmentation mechanism to obtain pseudo diverse views . |
| Outcome: | The proposed method minimizes intra-class variance by pulling together in-domain intents belonging to the same class and maximizes inter-class variation by pushing apart samples from different classes. |
Towards Open Environment Intent Prediction (2023.findings-acl)
Copied to clipboard
| Challenge: | Out-of-Domain (OOD) Intent Classification and New Intent Discovering are two tasks in the Task-Oriented Dialogue System. |
| Approach: | They propose a task paradigm to extend Out-of-Domain (OOD) Intent Classification and New Intent Discovering tasks in the Task-Oriented Dialogue System. |
| Outcome: | The proposed scheme improves on existing OOD intent classification and discovery datasets. |
Disentangled Knowledge Transfer for OOD Intent Discovery with Unified Contrastive Learning (2022.acl-short)
Copied to clipboard
| Challenge: | Existing methods to find out out-of-domain (OOD) intents do not take prior knowledge of in-domain data into account. |
| Approach: | They propose a disentangled knowledge transfer method to bridge the gap between IND pre-training and OOD clustering by using a unified multi-head contrastive learning framework. |
| Outcome: | The proposed method is able to group new unknown intents into different clusters, enabling future development of the system. |
Revisit Overconfidence for OOD Detection: Reassigned Contrastive Learning with Adaptive Class-dependent Threshold (2022.naacl-main)
Copied to clipboard
Yanan Wu, Keqing He, Yuanmeng Yan, QiXiang Gao, Zhiyuan Zeng, Fujia Zheng, Lulu Zhao, Huixing Jiang, Wei Wu, Weiran Xu
| Challenge: | Existing methods for OOD detection are based on labeled in-domain data . detecting out-of-domain (OOD) or unknown intents is challenging . |
| Approach: | They propose a novel reassigned contrastive learning method to discriminate IND intents for over-confident OOD and an adaptive class-dependent local threshold mechanism to separate similar IND and OOD intents. |
| Outcome: | The proposed method is effective for both aspects of overconfidence issues. |
Energy-based Unknown Intent Detection with Data Manipulation (2021.findings-acl)
Copied to clipboard
| Challenge: | Unknown intent detection aims to identify the out-of-distribution (OOD) utterance whose intent has never appeared in the training set. |
| Approach: | They propose a framework to generate high-quality OOD utterances with importance weighTs (GOT) their framework is fine-tuned to detect out-of-distribution utterrances . |
| Outcome: | The proposed framework can achieve state-of-the-art results on two benchmark datasets. |
Generalized Intent Discovery: Learning from Open World Dialogue System (2022.coling-1)
Copied to clipboard
| Challenge: | Existing intent classification models rely on a pre-defined intent set and supervised labels, which is limited in some practical scenarios. |
| Approach: | They propose to extend an IND intent classifier to an open-world intent set including IND and OOD intents. |
| Outcome: | The proposed task can classify IND and OOD intents while discovering new unlabeled OOD types incrementally. |