OoMMix: Out-of-manifold Regularization in Contextual Embedding Space for Text Classification (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies on neural networks with pre-trained weights focus on low-dimensional subspace where the embedding vectors computed from input words are located. |
| Approach: | They propose an approach to find and regularize the remainder of the space, referred to as out-of-manifold, which cannot be accessed through the words. |
| Outcome: | The proposed approach is able to fine-tune the out-of-manifold embedding space on text classification benchmarks. |
Similar Papers
Calibrated Language Model Fine-Tuning for In- and Out-of-Distribution Data (2020.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models suffer from severe miscalibration for both in-distribution and out-of-difference data due to over-parameterization. |
| Approach: | They propose a regularized method to improve in-distribution and out-of-distance calibrations by using on-manifold regularization and off-manfold regularisation. |
| Outcome: | The proposed method outperforms existing methods for text classification in terms of expectation calibration error, misclassification detection, and OOD detection on six datasets. |
Holistic Sentence Embeddings for Better Out-of-Distribution Detection (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained language models have shown superior performance for textual OOD detection, but they estimate sample distance scores in the last-layer CLS embedding space. |
| Approach: | They propose to use token averaging and layer combination to boost OOD detection by deriving more holistic sentence embeddings. |
| Outcome: | The proposed method surpasses the state-of-the-art on a comprehensive suite of benchmarks by a 9.33% FAR95 margin. |
Robust Backed-off Estimation of Out-of-Vocabulary Embeddings (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to solving out-of-vocabulary (OOV) words use subwords to represent oov words with a bag of subword. |
| Approach: | They propose a method to estimate oov word embeddings by referring to pre-trained word embeds for known words with similar surfaces to target ov words. |
| Outcome: | The proposed method improves word similarity tasks and biomedical tasks even with weak baselines. |
kFolden: k-Fold Ensemble for Out-Of-Distribution Detection (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies studying OOD detection in NLP often rely on external data to diversify model predictions. |
| Approach: | They propose a framework which mimics OOD detection behavior without external data . they take text classification as an archetype and compare them to existing datasets . |
| Outcome: | The proposed framework can resolve in- and out-distribution examples in a natural way using existing datasets. |
Out-of-Distribution Detection via LLM-Guided Outlier Generation for Text-attributed Graph (2025.findings-acl)
Copied to clipboard
| Challenge: | Text-Attributed Graphs (TAGs) are widely used in the real world. |
| Approach: | They propose to use Large Language Models to generate OOD-nodes with high quality . they also use LLMs to integrate existing nodes with LLM-generated edges . |
| Outcome: | The proposed method performs well on samples outside the In-Distribution (ID) data, but it is difficult to obtain high-quality OOD samples in the real world. |
Out-of-Distribution Generalization in Natural Language Processing: Past, Present, and Future (2023.emnlp-main)
Copied to clipboard
Linyi Yang, Yaoxian Song, Xuan Ren, Chenyang Lyu, Yidong Wang, Jingming Zhuo, Lingqiao Liu, Jindong Wang, Jennifer Foster, Yue Zhang
| Challenge: | Existing literature on the generalization of machine learning models to out-of-distribution data is lacking. |
| Approach: | They propose to present the first comprehensive review of recent progress, methods, and evaluations on the generalization challenge from an OOD perspective in natural language understanding. |
| Outcome: | The proposed survey provides the first comprehensive review of recent progress, methods, and evaluations on the generalization challenge from an OOD perspective in natural language understanding. |
Manifold Learning-based Word Representation Refinement Incorporating Global and Local Information (2020.coling-main)
Copied to clipboard
| Challenge: | Recent studies show word embedding models underestimate similarities between similar words and overestimate similarities between distant words. |
| Approach: | They propose two new word embedding methods that align original and re-fined embeddable spaces to a new refined semantic space. |
| Outcome: | The proposed methods outperform state-of-the-art methods for word representation refinement. |
SSMBA: Self-Supervised Manifold Based Data Augmentation for Improving Out-of-Domain Robustness (2020.emnlp-main)
Copied to clipboard
| Challenge: | Data augmentation is a common method used to improve out-of-domain (OOD) generalization. |
| Approach: | They propose a data augmentation method that uses corruption and reconstruction functions to move randomly on a manifold to generate training examples. |
| Outcome: | The proposed method outperforms existing methods and baseline models on both in-domain and OOD data and achieves gains of 0.8% on OOD Amazon reviews, 1.8% accuracy on OOO MNLI, and 1.4 BLEU on in- domain IWSLT14 German-English. |
Robust AI-Generated Text Detection by Restricted Embeddings (2024.findings-emnlp)
Copied to clipboard
Kristian Kuznetsov, Eduard Tulchinskii, Laida Kushnareva, German Magai, Serguei Barannikov, Sergey Nikolenko, Irina Piontkovskaya
| Challenge: | Existing approaches for artificial text detection are score-based and classifier-based . however, score-driven methods often rely on a score-derived score. |
| Approach: | They investigate the ability of classifier-based detectors to transfer to unseen generators or semantic domains. |
| Outcome: | The proposed methods improve the out-of-distribution classification score by up to 9% and 14%. |
Searching for the X-Factor: Exploring Corpus Subjectivity for Word Embeddings (P18-1)
Copied to clipboard
| Challenge: | Existing word embedding methods for natural language processing are limited in their ability to produce dense word embeds. |
| Approach: | They propose a word embedding SentiVec which is infused with sentiment information from a lexical resource and outperforms baselines on subjectivity-sensitive tasks. |
| Outcome: | The proposed word embedding SentiVec outperforms baselines on subjectivity-sensitive tasks. |