FLatS: Principled Out-of-Distribution Detection with Feature-Based Likelihood Ratio Score (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for detecting out-of-distribution instances are empirical . state-of the-art methods for OOD detection are suboptimal since they only estimate in-distance density pout(x). |
| Approach: | They propose a method that measures the “OOD-ness” of a test case x through the likelihood ratio between out-distribution mathcal Pout and in-division mathcal Pin. |
| Outcome: | The proposed method improves existing methods on popular benchmarks and establishes a new SOTA on popular NLP benchmarks. |
Similar Papers
kFolden: k-Fold Ensemble for Out-Of-Distribution Detection (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies studying OOD detection in NLP often rely on external data to diversify model predictions. |
| Approach: | They propose a framework which mimics OOD detection behavior without external data . they take text classification as an archetype and compare them to existing datasets . |
| Outcome: | The proposed framework can resolve in- and out-distribution examples in a natural way using existing datasets. |
On Prefix-tuning for Lightweight Out-of-distribution Detection (2023.acl-long)
Copied to clipboard
| Challenge: | Out-of-distribution (OOD) detection is a fundamental task vexing real-world applications . fine-tuning based methods require storing fine- tuned models for each scenario . |
| Approach: | They propose an unsupervised prefix-tuning based OOD detection framework called PTO . they propose to take advantage of optional training data labels and targeted OOD data . |
| Outcome: | The proposed framework performs better than existing methods under a wide range of metrics, detection settings, and OOD types. |
Out-of-Distribution Detection through Soft Clustering with Non-Negative Kernel Regression (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for detecting out-of-distribution data are computationally complex and storage-intensive. |
| Approach: | They propose a soft clustering approach for OOD detection based on non-negative kernel regression . their approach greatly reduces computational and space complexities while retaining competitive performance. |
| Outcome: | The proposed approach outperforms existing methods by up to 4 AUROC points on four benchmarks while retaining competitive performance. |
Classical Out-of-Distribution Detection Methods Benchmark in Text Classification Tasks (2023.acl-srw)
Copied to clipboard
| Challenge: | Current approaches to OOD detection in NLP are not yet sufficiently sensitive to capture all samples characterized by various types of distributional shifts. |
| Approach: | They evaluated eight methods that are easily integrable into existing NLP systems and require no additional OOD data or model modifications. |
| Outcome: | The proposed methods are easily integrable into existing NLP systems and require no additional OOD data or model modifications. |
Enhancing Out-of-Distribution Detection in Natural Language Understanding via Implicit Layer Ensemble (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Out-of-distribution (OOD) detection aims to discern outliers from the intended data distribution, which is crucial to maintaining high reliability and a good user experience. |
| Approach: | They propose a framework that encourages intermediate features to learn layer-specialized representations and assembles them implicitly into a single representation to absorb rich information in the pre-trained language model. |
| Outcome: | The proposed framework is significantly more effective than previous studies in intent classification and OOD datasets. |
Fine-Tuning Deteriorates General Textual Out-of-Distribution Detection by Distorting Task-Agnostic Features (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for detecting out-of-distribution inputs are underexplored . detecting semantic and non-semantic shifts is difficult for pre-tuned pre-trainers . |
| Approach: | They propose a general OOD score that integrates confidence scores from task-agnostic and task-specific representations to improve detecting semantic and non-semantic shifts. |
| Outcome: | The proposed method improves on two cross-task benchmarks with semantic and non-semantic shifts. |
How Good Are LLMs at Out-of-Distribution Detection? (2024.lrec-main)
Copied to clipboard
| Challenge: | Out-of-distribution (OOD) detection is crucial for ensuring AI safety . large language models (LLMs) are becoming more prevalent due to their scale, pre-training objectives, and paradigms used for inference. |
| Approach: | They propose to use large language models to investigate out-of-distribution (OOD) detection in machine learning. |
| Outcome: | The proposed method outperforms other OOD detectors in zero-grad and fine-tuning scenarios. |
SELFOOD: Self-Supervised Out-Of-Distribution Detection via Learning to Rank (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for supervised OOD detection require expensive manual annotation of in-distribution and OOD samples. |
| Approach: | They propose a self-supervised OOD detection method that requires only in-distribution samples as supervision. |
| Outcome: | Experiments with multiple classifiers on coarse- and fine-grained datasets show the proposed method performs well in both coarse-and fine-grid settings. |
Text Meets Topology: Rethinking Out-of-distribution Detection in Text-Rich Networks (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for out-of-distribution (OOD) detection ignore textual-structural diversity . text-rich networks (TrNs) represent complex interplay between textual content and relational structures . |
| Approach: | They propose a framework for evaluating out-of-distribution detection in text-rich networks . they propose augmentations, structural shifts, and domain-based divisions to model interplay . |
| Outcome: | Experiments on 11 datasets show the framework is effective in out-of-distribution detection. |
RainProof: An Umbrella to Shield Text Generator from Out-Of-Distribution Data (2023.emnlp-main)
Copied to clipboard
| Challenge: | Out-of-distribution (OOD) detection is a widely covered topic in classification tasks, but most methods rely on hidden features output by the encoder. |
| Approach: | They propose to leverage soft-probabilities in a black-box framework to detect OOD . they propose to use a more operational evaluation setting to enable OOD detection . |
| Outcome: | The proposed framework can access soft-predictions but not the internal states of the model. |