A Critical Analysis of Document Out-of-Distribution Detection (2023.findings-emnlp)
Copied to clipboard
Jiuxiang Gu, Yifei Ming, Yi Zhou, Jason Kuen, Vlad Morariu, Handong Zhao, Ruiyi Zhang, Nikolaos Barmpalios, Anqi Liu, Yixuan Li, Tong Sun, Ani Nenkova
| Challenge: | Existing document understanding models focus on single-modal inputs such as images or texts. |
| Approach: | They propose to use a spatial-aware adapter to adapt transformer-based language models to document domain to exploit multi-modal information. |
| Outcome: | The proposed model significantly improves the OOD detection performance compared to using a standard language model and to competitive baselines. |
Similar Papers
Exploring Large Language Models for Multi-Modal Out-of-Distribution Detection (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Out-of-distribution (OOD) detection is essential for reliable and trustworthy machine learning. |
| Approach: | They propose to apply world knowledge to enhance OOD detection performance through selective generation from large language models (LLMs) they propose to extract visual objects from each image to fully capitalize on the aforementioned world knowledge. |
| Outcome: | The proposed method outperforms the state-of-the-art on visual OOD detection on in-distribution (ID) samples. |
VI-OOD: A Unified Framework of Representation Learning for Textual Out-of-distribution Detection (2024.lrec-main)
Copied to clipboard
| Challenge: | Out-of-distribution (OOD) detection is a crucial part of deep neural networks. |
| Approach: | They propose a variational inference framework which maximizes the likelihood of the joint distribution p(x, y) instead of p[y|x). |
| Outcome: | The proposed framework maximizes the likelihood of the joint distribution p(x, y) instead of p[y|x). |
How Good Are LLMs at Out-of-Distribution Detection? (2024.lrec-main)
Copied to clipboard
| Challenge: | Out-of-distribution (OOD) detection is crucial for ensuring AI safety . large language models (LLMs) are becoming more prevalent due to their scale, pre-training objectives, and paradigms used for inference. |
| Approach: | They propose to use large language models to investigate out-of-distribution (OOD) detection in machine learning. |
| Outcome: | The proposed method outperforms other OOD detectors in zero-grad and fine-tuning scenarios. |
Enhancing Out-of-Distribution Detection in Natural Language Understanding via Implicit Layer Ensemble (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Out-of-distribution (OOD) detection aims to discern outliers from the intended data distribution, which is crucial to maintaining high reliability and a good user experience. |
| Approach: | They propose a framework that encourages intermediate features to learn layer-specialized representations and assembles them implicitly into a single representation to absorb rich information in the pre-trained language model. |
| Outcome: | The proposed framework is significantly more effective than previous studies in intent classification and OOD datasets. |
Classical Out-of-Distribution Detection Methods Benchmark in Text Classification Tasks (2023.acl-srw)
Copied to clipboard
| Challenge: | Current approaches to OOD detection in NLP are not yet sufficiently sensitive to capture all samples characterized by various types of distributional shifts. |
| Approach: | They evaluated eight methods that are easily integrable into existing NLP systems and require no additional OOD data or model modifications. |
| Outcome: | The proposed methods are easily integrable into existing NLP systems and require no additional OOD data or model modifications. |
Contrastive Out-of-Distribution Detection for Pretrained Transformers (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pretrained Transformers achieve remarkable performance when training and test data are from the same distribution, but in real-world scenarios, out-of-distribution instances can cause semantic shift problems. |
| Approach: | They propose to fine-tune the Transformers with a contrastive loss, which improves the compactness of representations, and to use the Mahalanobis distance in the model's penultimate layer to detect OOD instances. |
| Outcome: | The proposed method outperforms baselines in the real-world and achieves near-perfect OOD detection performance. |
Text Meets Topology: Rethinking Out-of-distribution Detection in Text-Rich Networks (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for out-of-distribution (OOD) detection ignore textual-structural diversity . text-rich networks (TrNs) represent complex interplay between textual content and relational structures . |
| Approach: | They propose a framework for evaluating out-of-distribution detection in text-rich networks . they propose augmentations, structural shifts, and domain-based divisions to model interplay . |
| Outcome: | Experiments on 11 datasets show the framework is effective in out-of-distribution detection. |
Types of Out-of-Distribution Texts and How to Detect Them (2021.emnlp-main)
Copied to clipboard
| Challenge: | Current NLP models produce unreliable or catastrophic predictions when training and test distributions differ . current models tend to produce unreliability or even catastrophic predictions that hurt user trust. |
| Approach: | They categorize examples as exhibiting a background shift or semantic shift and use calibration and density estimation methods to detect OOD examples. |
| Outcome: | The proposed methods beat calibration methods in background shift settings and perform worse in semantic shift settings. |
Out-of-Distribution Detection via LLM-Guided Outlier Generation for Text-attributed Graph (2025.findings-acl)
Copied to clipboard
| Challenge: | Text-Attributed Graphs (TAGs) are widely used in the real world. |
| Approach: | They propose to use Large Language Models to generate OOD-nodes with high quality . they also use LLMs to integrate existing nodes with LLM-generated edges . |
| Outcome: | The proposed method performs well on samples outside the In-Distribution (ID) data, but it is difficult to obtain high-quality OOD samples in the real world. |
RainProof: An Umbrella to Shield Text Generator from Out-Of-Distribution Data (2023.emnlp-main)
Copied to clipboard
| Challenge: | Out-of-distribution (OOD) detection is a widely covered topic in classification tasks, but most methods rely on hidden features output by the encoder. |
| Approach: | They propose to leverage soft-probabilities in a black-box framework to detect OOD . they propose to use a more operational evaluation setting to enable OOD detection . |
| Outcome: | The proposed framework can access soft-predictions but not the internal states of the model. |