DoLFIn: Distributions over Latent Features for Interpretability (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to interpret neural networks face a trade-off between a model's usefulness and its complexity. |
| Approach: | They propose a novel approach to achieve interpretability that avoids this trade-off by using probability as the central quantity instead of a fixed quantity. |
| Outcome: | The proposed approach outperforms the classical CNN and BiLSTM classifiers on the SST2 and AG-news datasets. |
Similar Papers
Interpretable Neural Predictions with Differentiable Binary Variables (P19-1)
Copied to clipboard
| Challenge: | Neural networks are bringing incredible performance gains on text classification tasks, but they also require interpretability. |
| Approach: | They propose a latent model that selects a rationale and a classifier that learns from the words in the rationale alone. |
| Outcome: | The proposed model can predict expected value of penalties without REINFORCE and can be directly optimised towards a pre-specified text selection rate. |
Generalizing Backpropagation for Gradient-Based Interpretability (2023.acl-long)
Copied to clipboard
| Challenge: | Several feature-attribution methods for interpreting deep neural networks rely on computing the gradients of a model’s output with respect to its inputs, but they reveal little about the inner workings of the model itself. |
| Approach: | They propose a generalized backpropagation algorithm that generalizes the gradient computation of a model to efficiently compute other interpretable statistics about the gradient graph of neural networks. |
| Outcome: | The proposed generalized algorithm can be used to compute other interpretable statistics about the gradient graph of a neural network, such as the highest-weighted path and entropy. |
Interpretability and Analysis in Neural NLP (2020.acl-tutorials)
Copied to clipboard
| Challenge: | a tutorial aims to introduce the nascent field of interpretability and analysis of neural networks in NLP . |
| Approach: | This tutorial will introduce the nascent field of interpretability and analysis of neural networks in NLP. |
| Outcome: | This tutorial will introduce the nascent field of interpretability and analysis of neural networks in NLP. |
SHAP-Based Explanation Methods: A Review for NLP Interpretability (2022.coling-1)
Copied to clipboard
| Challenge: | Existing models with opacity problems have been proposed to address this problem. |
| Approach: | They propose a unified local-interpretability framework with a rigorous theoretical foundation on the game-theoretic concept of Shapley values. |
| Outcome: | The proposed framework is based on the Shapley-value-based model explanations. |
Neuron-level Interpretation of Deep NLP Models: A Survey (2022.tacl-1)
Copied to clipboard
| Challenge: | Existing work on deep neural networks has focused on representation analysis, but recent work focused on analyzing neurons within these models. |
| Approach: | They propose to analyze neural networks to uncover linguistic concepts captured by the network . they propose to use a granular approach to analyze neurons within these models . |
| Outcome: | The proposed method combines methods to discover and understand neurons in a network with evaluation methods. |
Demystifying Neural Fake News via Linguistic Feature-Based Interpretation (2022.coling-1)
Copied to clipboard
| Challenge: | Recent advances to neural fake news generators have made it difficult to understand how misinformation generated by these models may best be confronted. |
| Approach: | They conduct feature-based analysis to gain an interpretative understanding of the linguistic attributes that neural fake news generators may most effectively exploit. |
| Outcome: | The proposed models are compared with models trained on subsets of features and confronted with increasingly advanced neural fake news. |
On the Lack of Robust Interpretability of Neural Text Classifiers (2021.findings-acl)
Copied to clipboard
Muhammad Bilal Zafar, Michele Donini, Dylan Slack, Cedric Archambeau, Sanjiv Das, Krishnaram Kenthapadi
| Challenge: | Several models have been proposed to interpret models with feature-based interpretability methods. |
| Approach: | They propose to quantify the robustness of neural text classifiers by using two randomization tests to compare models with identical initializations. |
| Outcome: | The proposed methods show surprising deviations from expected behavior . the results raise questions about the extent of insights that practitioners may draw from interpretations. |
RANCC: Rationalizing Neural Networks via Concept Clustering (2020.coling-main)
Copied to clipboard
| Challenge: | Existing models that construct explanations concurrently with classification predictions are opaque. |
| Approach: | They propose a self-explainable model for Natural Language Processing (NLP) text classification tasks . they extract a rationale from the text and use it to predict a concept of interest . |
| Outcome: | The proposed model can be compressed without complicated compression techniques. |
Instance-Based Neural Dependency Parsing (2021.tacl-1)
Copied to clipboard
Hiroki Ouchi, Jun Suzuki, Sosuke Kobayashi, Sho Yokoi, Tatsuki Kuribayashi, Masashi Yoshikawa, Kentaro Inui
| Challenge: | Existing models that use instance-based inference for dependency parsing are difficult to understand for humans. |
| Approach: | They develop neural models that adopt an interpretable inference process for dependency parsing. |
| Outcome: | The proposed models achieve competitive accuracy with standard neural models and have plausibility of instance-based explanations. |
RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representations (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods to disentangle individual neurons from multiple high-level concepts are not yet benchmarked. |
| Approach: | They propose a method of Multi-task Distributed Alignment Search that allows to find distributed representations satisfying multiple causal criteria. |
| Outcome: | The proposed method achieves state-of-the-art on the target language model with Llama2-7B . |