Triple-Hybrid Energy-based Model Makes Better Calibrated Natural Language Understanding Models (2023.eacl-main)
Copied to clipboard
| Challenge: | In-distribution (ID) miscalibration and out-of-difference (OOD) detection are main concerns for pre-trained language models. |
| Approach: | They propose a triple-hybrid EBM which combines the benefits of classifier, conditional generative model and marginal generative models altogether. |
| Outcome: | The proposed model outperforms previous methods in terms of ID calibration and OOD detection by a large margin while maintaining competitive accuracy. |
Similar Papers
Joint Energy-based Model Training for Better Calibrated Natural Language Understanding Models (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing calibration methods rescale posterior distributions of classifiers after training. |
| Approach: | They propose to use a noise contrastive estimation technique to train an energy-based model during finetuning of pretrained text encoders. |
| Outcome: | The proposed model can reach a better calibration competitive to strong baselines with little or no loss in accuracy. |
Enhancing Out-of-Distribution Detection in Natural Language Understanding via Implicit Layer Ensemble (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Out-of-distribution (OOD) detection aims to discern outliers from the intended data distribution, which is crucial to maintaining high reliability and a good user experience. |
| Approach: | They propose a framework that encourages intermediate features to learn layer-specialized representations and assembles them implicitly into a single representation to absorb rich information in the pre-trained language model. |
| Outcome: | The proposed framework is significantly more effective than previous studies in intent classification and OOD datasets. |
Exploring Large Language Models for Multi-Modal Out-of-Distribution Detection (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Out-of-distribution (OOD) detection is essential for reliable and trustworthy machine learning. |
| Approach: | They propose to apply world knowledge to enhance OOD detection performance through selective generation from large language models (LLMs) they propose to extract visual objects from each image to fully capitalize on the aforementioned world knowledge. |
| Outcome: | The proposed method outperforms the state-of-the-art on visual OOD detection on in-distribution (ID) samples. |
Out-of-Distribution Generalization in Natural Language Processing: Past, Present, and Future (2023.emnlp-main)
Copied to clipboard
Linyi Yang, Yaoxian Song, Xuan Ren, Chenyang Lyu, Yidong Wang, Jingming Zhuo, Lingqiao Liu, Jindong Wang, Jennifer Foster, Yue Zhang
| Challenge: | Existing literature on the generalization of machine learning models to out-of-distribution data is lacking. |
| Approach: | They propose to present the first comprehensive review of recent progress, methods, and evaluations on the generalization challenge from an OOD perspective in natural language understanding. |
| Outcome: | The proposed survey provides the first comprehensive review of recent progress, methods, and evaluations on the generalization challenge from an OOD perspective in natural language understanding. |
Large Language Models for Anomaly and Out-of-Distribution Detection: A Survey (2025.findings-naacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated their effectiveness in natural language processing but also in broader applications due to their advanced comprehension and generative capabilities. |
| Approach: | They propose a taxonomy to categorize existing approaches into two classes based on the role played by LLMs. |
| Outcome: | The proposed taxonomy categorizes existing approaches into two classes based on the role played by LLMs. |
Heterogeneity over Homogeneity: Investigating Multilingual Speech Pre-Trained Models for Detecting Audio Deepfake (2024.findings-naacl)
Copied to clipboard
| Challenge: | a recent study has focused on audio deepfake detection (ADD) due to its ability to impersonate and share false, often malicious information. |
| Approach: | They propose to use multilingual speech Pre-Trained models for Audio deepfake detection (ADD) they propose to combine models with existing models to achieve better ADD detection . |
| Outcome: | The proposed models gain knowledge about diverse pitches, accents, and tones, during theirpre-training phase and are more robust to variations. |
GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-Distribution Generalization Perspective (2023.findings-acl)
Copied to clipboard
Linyi Yang, Shuibai Zhang, Libo Qin, Yafu Li, Yidong Wang, Hanmeng Liu, Jindong Wang, Xing Xie, Yue Zhang
| Challenge: | Pre-trained language models (PLMs) have improved generalization performance but the out-of-distribution (OOD) generalization problem remains a challenge in many NLP tasks. |
| Approach: | They propose to create a benchmark for evaluating out-of-distribution (OOD) generalization in NLP models. |
| Outcome: | The proposed benchmarks highlight the importance of OOD robustness and provide insights on how to measure it and improve it. |
DMHM: Density-aware Manifold Learning and Hybrid Mahalanobis Energy for LLMs-generated Text Detection (2026.acl-long)
Copied to clipboard
Tianle Liu, Zhiliang Tian, Zhen Huang, Tianlun Liu, Jingyuan Huang, Zhaoning Zhang, Chengcheng Shao, Dongsheng Li
| Challenge: | Existing methods for LGT detection assume that it is a single homogeneous distribution. |
| Approach: | They propose a framework for LGT detection based on density-aware manifold learning and hybrid Mahalanobis energy. |
| Outcome: | The proposed framework outperforms baselines in detecting LLM-generated text (LGT) it is based on density-aware manifold learning and hybrid Mahalanobis energy . |
Extrapolating Multilingual Understanding Models as Multilingual Generators (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing multilingual understanding models are not capable of generating high-quality text compared with decoder-based causal language models. |
| Approach: | They propose a method to adapt a multilingual encoder to a language generator with a small number of additional parameters. |
| Outcome: | The proposed approach outperforms initialization-based methods with 9.4 BLEU on machine translation, 8.1 Rouge-L on question generation, and 5.5 METEOR on story generation. |
Rule Discovery for Natural Language Inference Data Generation Using Out-of-Distribution Detection (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing training rules for natural language inference do not cover the diversity of natural language. |
| Approach: | They propose a framework that combines out-of-distribution detection and clustering to identify new premise–hypothesis pairs in a dataset that are not covered by existing rules. |
| Outcome: | The proposed framework achieves +0.85%p accuracy on 2k and +0.15%p on 550k samples. |