Interventional Training for Out-Of-Distribution Natural Language Understanding (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for NLU training use only known and single confounders, but in many NLU tasks the confounder can be unknown and multifactorial. |
| Approach: | They propose a method that performs multi-granular intervention with identified multifactorial confounders by using a bottom-up automatic intervention method. |
| Outcome: | The proposed method performs multi-granular intervention with identified multifactorial confounders on three NLU tasks, namely, natural language inference, fact verification and paraphrase identification. |
Similar Papers
End-to-End Self-Debiasing Framework for Robust NLU Training (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing models incorporate dataset biases leading to strong performance on in-distribution test sets but poor performance on out-of-distortion (OOD) tests. |
| Approach: | They propose a debiasing framework where the shallow representations of the main model are used to derive a bias model and both models are trained simultaneously. |
| Outcome: | The proposed framework outperforms existing approaches on three well-studied NLU tasks while still delivering high in-distribution performance. |
Out-of-Distribution Generalization in Natural Language Processing: Past, Present, and Future (2023.emnlp-main)
Copied to clipboard
Linyi Yang, Yaoxian Song, Xuan Ren, Chenyang Lyu, Yidong Wang, Jingming Zhuo, Lingqiao Liu, Jindong Wang, Jennifer Foster, Yue Zhang
| Challenge: | Existing literature on the generalization of machine learning models to out-of-distribution data is lacking. |
| Approach: | They propose to present the first comprehensive review of recent progress, methods, and evaluations on the generalization challenge from an OOD perspective in natural language understanding. |
| Outcome: | The proposed survey provides the first comprehensive review of recent progress, methods, and evaluations on the generalization challenge from an OOD perspective in natural language understanding. |
Enhancing Out-of-Distribution Detection in Natural Language Understanding via Implicit Layer Ensemble (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Out-of-distribution (OOD) detection aims to discern outliers from the intended data distribution, which is crucial to maintaining high reliability and a good user experience. |
| Approach: | They propose a framework that encourages intermediate features to learn layer-specialized representations and assembles them implicitly into a single representation to absorb rich information in the pre-trained language model. |
| Outcome: | The proposed framework is significantly more effective than previous studies in intent classification and OOD datasets. |
Unsupervised training data re-weighting for natural language understanding with local distribution approximation (2022.emnlp-industry)
Copied to clipboard
| Challenge: | a distribution mismatch between offline training and live data can cause biases . cyclic seasonality shifts, and changing pool of users can contribute to this problem . |
| Approach: | They propose an unsupervised approach to mitigate offline training data sampling bias . they propose a local distribution approximation in the pre-trained embedding space . |
| Outcome: | The proposed approach mitigates the offline training data sampling bias in multiple NLU tasks without additional annotation. |
GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-Distribution Generalization Perspective (2023.findings-acl)
Copied to clipboard
Linyi Yang, Shuibai Zhang, Libo Qin, Yafu Li, Yidong Wang, Hanmeng Liu, Jindong Wang, Xing Xie, Yue Zhang
| Challenge: | Pre-trained language models (PLMs) have improved generalization performance but the out-of-distribution (OOD) generalization problem remains a challenge in many NLP tasks. |
| Approach: | They propose to create a benchmark for evaluating out-of-distribution (OOD) generalization in NLP models. |
| Outcome: | The proposed benchmarks highlight the importance of OOD robustness and provide insights on how to measure it and improve it. |
Rule Discovery for Natural Language Inference Data Generation Using Out-of-Distribution Detection (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing training rules for natural language inference do not cover the diversity of natural language. |
| Approach: | They propose a framework that combines out-of-distribution detection and clustering to identify new premise–hypothesis pairs in a dataset that are not covered by existing rules. |
| Outcome: | The proposed framework achieves +0.85%p accuracy on 2k and +0.15%p on 550k samples. |
End-to-End Bias Mitigation by Modelling Biases in Corpora (2020.acl-main)
Copied to clipboard
| Challenge: | Recent studies have shown that strong natural language understanding models are prone to relying on unwanted dataset biases without learning the underlying task. |
| Approach: | They propose two learning strategies to train neural models that are more robust to dataset biases and transfer better to out-of-domain datasets. |
| Outcome: | The proposed methods improve robustness in all settings and transfer better to out-of-domain datasets. |
Mind the Trade-off: Debiasing NLU Models without Degrading the In-distribution Performance (2020.acl-main)
Copied to clipboard
| Challenge: | Recent studies show that pre-trained language models rely heavily on idiosyncratic biases of datasets. |
| Approach: | They propose a method which discourages models from exploiting biases while enabling them to receive enough incentive to learn from all the training examples. |
| Outcome: | The proposed method improves on out-of-distribution datasets while maintaining original in-district accuracy. |
Contrastive Novelty-Augmented Learning: Anticipating Outliers with Large Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for classification are overly confident on unseen examples . despite recent advances in NLP, some categories of distribution shift still pose serious challenges. |
| Approach: | They propose a method that generates OOD examples representative of novel classes and trains to decrease confidence on them. |
| Outcome: | The proposed method improves classifiers' ability to detect and abstain on novel class examples over previous methods by 2.3% and 5.5% over previous approaches. |
Out-of-Distribution Detection via LLM-Guided Outlier Generation for Text-attributed Graph (2025.findings-acl)
Copied to clipboard
| Challenge: | Text-Attributed Graphs (TAGs) are widely used in the real world. |
| Approach: | They propose to use Large Language Models to generate OOD-nodes with high quality . they also use LLMs to integrate existing nodes with LLM-generated edges . |
| Outcome: | The proposed method performs well on samples outside the In-Distribution (ID) data, but it is difficult to obtain high-quality OOD samples in the real world. |