| Challenge: | Interpretability of deep neural networks has gained a lot of attention in recent years, especially in NLP, where state-of-the-art models are being widely deployed and used in practice. |
| Approach: | They propose to analyze what linguistic and non-linguistic knowledge is learned within deep neural networks and highlight the salient parts of the input. |
| Outcome: | The proposed tool is useful for debugging, unraveling model bias, and for highlighting spurious correlations in a model. |
Similar Papers
NeuroX Library for Neuron Analysis of Deep NLP Models (2023.acl-demo)
Copied to clipboard
| Challenge: | NeuroX is an open-source toolkit to conduct neuron analysis of natural language processing models. |
| Approach: | They propose a Python toolkit to conduct neuron analysis of natural language processing models. |
| Outcome: | a new open-source toolkit enables neuron analysis of natural language processing models . the framework provides a framework for data processing and evaluation, making it easier for researchers and practitioners to perform neuron analyses. |
SELFEXPLAIN: A Self-Explaining Architecture for Neural Text Classifiers (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing models that explain text classification predictions are opaque and overfit to spurious artifacts. |
| Approach: | They propose a novel self-explaining model that explains a text classifier’s predictions using phrase-based concepts. |
| Outcome: | The proposed model shows that it is adequate, trustworthy and understandable by human judges compared to existing baselines. |
Latent Structure Models for Natural Language Processing (P19-4)
Copied to clipboard
| Challenge: | Latent structure models are a powerful tool for compositional data modeling and pipelines. |
| Approach: | This tutorial will cover recent advances in discrete latent structure models . it will discuss their motivation, potential, and limitations . |
| Outcome: | This tutorial will cover recent advances in discrete latent structure models . it will discuss their motivation, potential, and limitations . |
Latent Concept-based Explanation of NLP Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing attempts to explain deep learning models rely on input features, such as the words . however, such explanations are often less informative due to the discrete nature of words and lack of contextual verbosity. |
| Approach: | They propose a method that generates explanations for predictions based on latent concepts . they map the representations of salient input words into the training latent space . |
| Outcome: | The proposed method generates explanations for predictions based on latent concepts . it maps representations of salient input words into training latent space . |
Deep Latent Variable Models of Natural Language (D18-3)
Copied to clipboard
| Challenge: | In this tutorial, we will discuss the challenges of applying neural variational inference to NLP problems. |
| Approach: | The tutorial will cover deep latent variable models in the case where exact inference over the latent variables is tractable. |
| Outcome: | The proposed tutorial will cover deep latent variable models in the case where inference cannot be performed tractably and when it is not . |
Neuron-level Interpretation of Deep NLP Models: A Survey (2022.tacl-1)
Copied to clipboard
| Challenge: | Existing work on deep neural networks has focused on representation analysis, but recent work focused on analyzing neurons within these models. |
| Approach: | They propose to analyze neural networks to uncover linguistic concepts captured by the network . they propose to use a granular approach to analyze neurons within these models . |
| Outcome: | The proposed method combines methods to discover and understand neurons in a network with evaluation methods. |
On the Transformation of Latent Space in Fine-Tuned NLP Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | a large body of work analyzed the knowledge learned within representations of pre-trained models. |
| Approach: | They use hierarchical clustering to discover latent concepts in representational space . they compare pre-trained and fine-tuned models and perform a thorough analysis . |
| Outcome: | The results show that the model space evolves towards task-specific concepts whereas the lower layers retain generic concepts acquired in the pre-trained model. |
CNNs for NLP in the Browser: Client-Side Deployment and Visualization Opportunities (N18-5)
Copied to clipboard
| Challenge: | a JavaScript implementation of a convolutional neural network performs feedforward inference completely in the browser. |
| Approach: | They propose a JavaScript implementation that performs feedforward inference completely in the browser. |
| Outcome: | The proposed model performs feedforward inference completely in the browser without server requests . the proposed model is useful for applications with stringent latency requirements or low connectivity . |
Unsupervised Deep Structured Semantic Models for Commonsense Reasoning (N19-1)
Copied to clipboard
| Challenge: | Existing methods for commonsense reasoning rely on human-crafted features and knowledge bases, but unsupervised learning is not feasible due to the lack of labeled training data or comprehensive knowledge bases. |
| Approach: | They propose two unsupervised models based on the Deep Structured Semantic Models framework to tackle two commonsense reasoning tasks: Winograd Schema Challenge (WSC) and Pronoun Disambiguation (PDP). |
| Outcome: | The proposed models capture contextual information in the sentence and co-reference information between pronouns and nouns, and achieve significant improvement over previous state-of-the-art approaches. |
NeuronBlocks: Building Your NLP DNN Models Like Playing Lego (D19-3)
Copied to clipboard
| Challenge: | Deep Neural Networks (DNN) have been widely employed in industry to address various natural language processing tasks. |
| Approach: | They propose an NLP toolkit that encapsulates neural network modules as building blocks to construct various DNN models with complex architecture. |
| Outcome: | The proposed toolkit can build, train, and test various DNN models with complex architecture. |