| Challenge: | Existing models are limited in the number of available datasets and lack the necessary tools to improve them. |
| Approach: | They propose a framework which enables humans to debug deep learning text classifiers by disabling irrelevant hidden features. |
| Outcome: | Experiments show that using FIND, humans can improve CNN text classifiers trained on different types of imperfect datasets. |
Similar Papers
Adversarial NLI: A New Benchmark for Natural Language Understanding (2020.acl-main)
Copied to clipboard
| Challenge: | a new large-scale NLI benchmark dataset is presented to test models on a variety of popular NLIs. |
| Approach: | They propose a large-scale NLI benchmark dataset that is iteratively compared with a human-and-model-in-the-loop procedure. |
| Outcome: | The proposed method can be applied in a never-ending learning scenario, becoming a moving target for NLU, rather than a static benchmark that will quickly saturate. |
Investigating Ensemble Methods for Model Robustness Improvement of Text Classifiers (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to reduce model's reliance on bias features ignore the learnability of these features. |
| Approach: | They propose to reduce models' reliance on bias features by first training models with fixed low-capacity models which ignore the learnability of the bias features. |
| Outcome: | The proposed models can perform better on out-of-distribution datasets than baseline models with a more sophisticated model design. |
Topics to Avoid: Demoting Latent Confounds in Text Classification (D19-1)
Copied to clipboard
| Challenge: | Despite impressive performance on many text classification tasks, deep neural networks tend to learn frequent superficial patterns that are specific to the training data and do not always generalize well. |
| Approach: | They propose a method that represents latent topical confounds and a model which “unlearns” confounding features by predicting both the label of the input text and the confound. |
| Outcome: | The proposed model generalizes better and learns features indicative of the writing style rather than the content. |
The Authors Matter: Understanding and Mitigating Implicit Bias in Deep Text Classification (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on text classification have focused on the bias towards the individuals mentioned in the text content. |
| Approach: | They propose a framework to mitigate implicit bias in text classification models based on demographic attributes of authors . they propose to use this framework to train deep text classifiers to make predictions on the right features . |
| Outcome: | The proposed framework outperforms existing models significantly in fairness and performance. |
When and Why Does Bias Mitigation Work? (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Neural models exploit shallow surface features to perform language understanding tasks, rather than learning the deeper language understanding and reasoning skills that practitioners desire. |
| Approach: | They propose to use model debiasing techniques to pressure models away from spurious features and to use them to learn useful representations instead. |
| Outcome: | The proposed methods increase models' reliance on hidden biases instead of learning robust features that help them solve a task. |
Outlier-Aware Training for Improving Group Accuracy Disparities (2022.aacl-srw)
Copied to clipboard
| Challenge: | Methods addressing spurious correlations such as Just Train Twice involve reweighting a subset of the training set to maximize the worst-group accuracy. |
| Approach: | They propose to reweight a subset of a training set to maximize the worst-group accuracy by detecting outliers and removing them before reweighing. |
| Outcome: | The proposed method achieves competitive or better accuracy compared with JTT and can detect and remove annotation errors in the subset being reweighted in JTT. |
Detecting Machine-Generated Text: Techniques and Challenges (2024.acl-tutorials)
Copied to clipboard
| Challenge: | This tutorial focuses on machine-generated text and deepfakes. |
| Approach: | This tutorial aims to provide a comprehensive overview of text detection techniques . it will focus on machine-generated text and deepfakes . |
| Outcome: | This tutorial focuses on machine-generated text and deepfakes. |
Adversarial Removal of Demographic Attributes from Text Data (D18-1)
Copied to clipboard
| Challenge: | Recent advances in Representation Learning and Adversarial Training remove unwanted features from the learned representation. |
| Approach: | They show that demographic information of authors is encoded in the intermediate representations learned by text-based neural classifiers. |
| Outcome: | The proposed approach achieves higher accuracies on the same dataset, the authors show . they show that the proposed approach is effective in removing unwanted features from the learned representations. |
Analyzing Biases to Spurious Correlations in Text Classification Tasks (2022.aacl-short)
Copied to clipboard
| Challenge: | Often these systems exceed human performance, but there is a caveat: standard benchmarks often assume that training and evaluation data are drawn independently and identically from the same underlying distribution. |
| Approach: | They propose to exploit spurious correlations in training data to exploit these correlations . they show that even when only ‘stop’ words are available, it is possible to predict the class significantly better than random. |
| Outcome: | The proposed model can predict class significantly better when only ‘stop’ words are available at the input stage, but can degrade the ability of the system to generalize well to out-of-domain data. |
CURE: Controlled Unlearning for Robust Embeddings — Mitigating Conceptual Shortcuts in Pre-Trained Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained language models are susceptible to spurious, concept-driven correlations that impair robustness and fairness. |
| Approach: | They propose a framework that disentangles and suppresses conceptual shortcuts while preserving essential content information. |
| Outcome: | The proposed framework improves on IMDB and Yelp datasets with minimal computational overhead. |