| Challenge: | Out-of-distribution (OOD) detection is crucial for ensuring AI safety . large language models (LLMs) are becoming more prevalent due to their scale, pre-training objectives, and paradigms used for inference. |
| Approach: | They propose to use large language models to investigate out-of-distribution (OOD) detection in machine learning. |
| Outcome: | The proposed method outperforms other OOD detectors in zero-grad and fine-tuning scenarios. |
Similar Papers
Large Language Models for Anomaly and Out-of-Distribution Detection: A Survey (2025.findings-naacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated their effectiveness in natural language processing but also in broader applications due to their advanced comprehension and generative capabilities. |
| Approach: | They propose a taxonomy to categorize existing approaches into two classes based on the role played by LLMs. |
| Outcome: | The proposed taxonomy categorizes existing approaches into two classes based on the role played by LLMs. |
Out-of-Distribution Detection via LLM-Guided Outlier Generation for Text-attributed Graph (2025.findings-acl)
Copied to clipboard
| Challenge: | Text-Attributed Graphs (TAGs) are widely used in the real world. |
| Approach: | They propose to use Large Language Models to generate OOD-nodes with high quality . they also use LLMs to integrate existing nodes with LLM-generated edges . |
| Outcome: | The proposed method performs well on samples outside the In-Distribution (ID) data, but it is difficult to obtain high-quality OOD samples in the real world. |
Navigating the Unknown: Intent Classification and Out-of-Distribution Detection Using Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Out-of-Distribution (OOD) detection requires great generalization capability . |
| Approach: | They propose a method that is cost-efficient, high-performing, highly robust and versatile enough to be used with smaller LLMs without sacrificing performance. |
| Outcome: | The proposed method is cost-efficient, high-performing, robust, and versatile enough to be used with smaller LLMs without sacrificing performance. |
Exploring Large Language Models for Multi-Modal Out-of-Distribution Detection (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Out-of-distribution (OOD) detection is essential for reliable and trustworthy machine learning. |
| Approach: | They propose to apply world knowledge to enhance OOD detection performance through selective generation from large language models (LLMs) they propose to extract visual objects from each image to fully capitalize on the aforementioned world knowledge. |
| Outcome: | The proposed method outperforms the state-of-the-art on visual OOD detection on in-distribution (ID) samples. |
Controlling Out-of-Domain Gaps in LLMs for Genre Classification and Generated Text Detection (2025.coling-main)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) have pushed the boundaries of natural language processing, but their consistency is often limited when applied to unfamiliar domains. |
| Approach: | They propose a method that controls which predictive indicators are used and which are excluded during classification. |
| Outcome: | The proposed method reduces the OOD gap by up to 20 percentage points in a few-shot setup. |
Is Fine-tuning Needed? Pre-trained Language Models Are Near Perfect for Out-of-Domain Detection (2023.acl-long)
Copied to clipboard
| Challenge: | Out-of-distribution (OOD) detection is critical for reliable predictions over text . fine-tuning with pre-trained language models has been a de facto procedure . |
| Approach: | They propose to leverage pre-trained language models for OOD detection without fine-tuning on ID data. |
| Outcome: | The proposed approach outperforms the fine-tuned model under distributional shifts. |
Beyond the Known: Investigating LLMs Performance on Out-of-Domain Intent Detection (2024.lrec-main)
Copied to clipboard
Pei Wang, Keqing He, Yejie Wang, Xiaoshuai Song, Yutao Mou, Jingang Wang, Yunsen Xian, Xunliang Cai, Weiran Xu
| Challenge: | Out-of-domain (OOD) intent detection is crucial for task-oriented dialogue systems. |
| Approach: | They conduct a comprehensive evaluation of large language models (LLMs) under various experimental settings and outline their strengths and weaknesses. |
| Outcome: | The proposed models exhibit strong zero-shot and few-shot capabilities, but is still at a disadvantage compared to models fine-tuned with full resource. |
Fine-Tuning Deteriorates General Textual Out-of-Distribution Detection by Distorting Task-Agnostic Features (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for detecting out-of-distribution inputs are underexplored . detecting semantic and non-semantic shifts is difficult for pre-tuned pre-trainers . |
| Approach: | They propose a general OOD score that integrates confidence scores from task-agnostic and task-specific representations to improve detecting semantic and non-semantic shifts. |
| Outcome: | The proposed method improves on two cross-task benchmarks with semantic and non-semantic shifts. |
Out-of-Distribution Generalization in Natural Language Processing: Past, Present, and Future (2023.emnlp-main)
Copied to clipboard
Linyi Yang, Yaoxian Song, Xuan Ren, Chenyang Lyu, Yidong Wang, Jingming Zhuo, Lingqiao Liu, Jindong Wang, Jennifer Foster, Yue Zhang
| Challenge: | Existing literature on the generalization of machine learning models to out-of-distribution data is lacking. |
| Approach: | They propose to present the first comprehensive review of recent progress, methods, and evaluations on the generalization challenge from an OOD perspective in natural language understanding. |
| Outcome: | The proposed survey provides the first comprehensive review of recent progress, methods, and evaluations on the generalization challenge from an OOD perspective in natural language understanding. |
Improving the OOD Performance of Closed-Source LLMs on NLI Through Strategic Data Selection (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods to improve robustness require changing the fine-tuning process or large-scale data augmentation, which are infeasible or cost prohibitive for closed-source models. |
| Approach: | They propose to prioritize more complex examples or replace existing training examples with LLM-generated data to improve performance on OOD NLI datasets. |
| Outcome: | The proposed methods improve performance on difficult OOD datasets while training with synthetic data leads to substantial improvements on easier OOD data. |