| Challenge: | Existing methods for NLP use input reduction to determine a word's importance . human accuracy degrades when shown the reduced examples instead of the original . |
| Approach: | They propose a process that iteratively removes the least important word from an input . they show human models make the same predictions with high confidence . |
| Outcome: | The proposed methods expose pathological behaviors of neural models . human experiments show that reduced examples lack information to support the prediction of any label . |
Similar Papers
Revisiting Pathologies of Neural Models under Input Reduction (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have shown that modern neural models tend to be miscalibrated. |
| Approach: | They examine why models produce high-confidence predictions on inputs that appear nonsensical to humans . previous work suggested that models fail to assign low probabilities due to model overconfidence . |
| Outcome: | The proposed methods can be extended to reduce the number of examples but with the cost of miscalibration. |
Interpreting Predictions of NLP Models (2020.emnlp-tutorials)
Copied to clipboard
| Challenge: | This tutorial will provide a background on interpretation techniques for neural NLP models. |
| Approach: | This tutorial will provide a background on interpretation techniques for NLP models . it will examine saliency maps, input perturbations, adversarial attacks and influence functions . |
| Outcome: | This tutorial will provide a background on interpretation techniques . examples-specific interpretations include saliency maps, input perturbations, adversarial attacks, influence functions . |
Interpretability and Analysis in Neural NLP (2020.acl-tutorials)
Copied to clipboard
| Challenge: | a tutorial aims to introduce the nascent field of interpretability and analysis of neural networks in NLP . |
| Approach: | This tutorial will introduce the nascent field of interpretability and analysis of neural networks in NLP. |
| Outcome: | This tutorial will introduce the nascent field of interpretability and analysis of neural networks in NLP. |
Fine-grained Interpretation and Causation Analysis in Deep NLP Models (2021.naacl-tutorials)
Copied to clipboard
| Challenge: | Despite the proven efficacy of deep neural networks at-large, their opaqueness is a major cause of concern. |
| Approach: | They will present research work on interpreting fine-grained components of a neural network model from two perspectives, i) fine-grain interpretation, and ii) causation analysis. |
| Outcome: | This paper presents work on interpreting fine-grained components of a neural network model from two perspectives, i) fine-grain interpretation, and ii) causation analysis. |
AllenNLP Interpret: A Framework for Explaining Predictions of NLP Models (D19-3)
Copied to clipboard
| Challenge: | Existing interpretation codebases make it difficult to apply these methods to new models and tasks. |
| Approach: | They propose a framework for interpreting NLP models that provides explanations for specific models. |
| Outcome: | The proposed framework provides interpretation primitives for any AllenNLP model and task, a suite of built-in interpretation methods, and a library of front-end visualization components. |
Neuron-level Interpretation of Deep NLP Models: A Survey (2022.tacl-1)
Copied to clipboard
| Challenge: | Existing work on deep neural networks has focused on representation analysis, but recent work focused on analyzing neurons within these models. |
| Approach: | They propose to analyze neural networks to uncover linguistic concepts captured by the network . they propose to use a granular approach to analyze neurons within these models . |
| Outcome: | The proposed method combines methods to discover and understand neurons in a network with evaluation methods. |
Interpretation of NLP models through input marginalization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to interpret NLP predictions replace each token with a predefined value, resulting in misleading interpretations. |
| Approach: | They propose to marginalize each token out of the training data distribution to demystify the "black box" property of deep neural networks for natural language processing. |
| Outcome: | The proposed method marginalizes each token out of the training data distribution. |
Deep Neural Model Inspection and Comparison via Functional Neuron Pathways (P19-1)
Copied to clipboard
| Challenge: | a general method for the interpretation and comparison of neural models is proposed . we factor a complex neural model into its functional components . |
| Approach: | They propose a method that factored a complex neural model into its functional components . they use correlated task level and linguistic heuristics to identify correlated pathways . |
| Outcome: | The proposed method can be applied in a purely post-processing manner to understand neural models. |
Using Interpretation Methods for Model Enhancement (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing frameworks for enhancing neural models with interpretation methods and gold rationales have not been fully explored. |
| Approach: | They propose a framework for utilizing interpretation methods and gold rationales to enhance neural models. |
| Outcome: | The proposed framework outperforms gradient-based methods in low-resource settings on a variety of tasks. |
Mitigating the Inconsistency Between Word Saliency and Model Confidence with Pathological Contrastive Training (2022.findings-acl)
Copied to clipboard
| Challenge: | Neural networks are used for various NLP tasks, but their complexity makes them difficult to interpret. |
| Approach: | They propose a framework to mitigate the model pathology and obtain more interpretable models by using contrastive learning and saliency-based samples augmentation to calibrate the sentences representation. |
| Outcome: | The proposed framework can mitigate the model pathology and generate more interpretable models while keeping the model performance. |