Towards Faithfully Interpretable NLP Systems: How Should We Define and Evaluate Faithfulness? (2020.acl-main)
Copied to clipboard
| Challenge: | Current approaches to interpretability evaluation focus on faithfulness criteria . current approaches focus on readability, plausibility and faithfulness . |
| Approach: | They argue that current binary definition of faithfulness sets unrealistic standards . they argue that a more graded definition would be of greater practical utility . |
| Outcome: | The proposed approach is based on three assumptions and lacks a graded definition of faithfulness. |
Similar Papers
A Comparative Study of Faithfulness Metrics for Model Interpretability Methods (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods to reveal the reasoning processes of machine learning models are difficult to interpret due to their complexity. |
| Approach: | They propose to use diagnosticity and complexity to assess faithfulness of machine learning models . they propose to apply posthoc interpretation methods to reveal reasoning behind models based on internal reasoning . |
| Outcome: | The proposed interpretation metrics show conflicting preferences when comparing interpretations . sufficiency and comprehensiveness metrics have higher diagnosticity and lower complexity . |
Interpretability and Analysis in Neural NLP (2020.acl-tutorials)
Copied to clipboard
| Challenge: | a tutorial aims to introduce the nascent field of interpretability and analysis of neural networks in NLP . |
| Approach: | This tutorial will introduce the nascent field of interpretability and analysis of neural networks in NLP. |
| Outcome: | This tutorial will introduce the nascent field of interpretability and analysis of neural networks in NLP. |
On the Sensitivity and Stability of Model Interpretations in NLP (2022.acl-long)
Copied to clipboard
| Challenge: | Recent years have witnessed the emergence of post-hoc interpretations that aim to uncover how NLP models make predictions. |
| Approach: | They propose two new criteria that provide complementary notions of faithfulness to removal-based criteria. |
| Outcome: | The proposed methods overcome limitations of gradient-based methods on removal-based criteria and overcome limitations in the proposed methods. |
On Sample Based Explanation Methods for NLP: Faithfulness, Efficiency and Semantic Evaluation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for explaining "black-box" models such as Influence Functions are becoming more popular. |
| Approach: | They propose a semantic-based evaluation metric that can better align with humans’ judgment of explanations than the widely adopted diagnostic or re-training measures. |
| Outcome: | The proposed method can better align with humans’ judgment of explanations than diagnostic or re-training measures. |
Neuron-level Interpretation of Deep NLP Models: A Survey (2022.tacl-1)
Copied to clipboard
| Challenge: | Existing work on deep neural networks has focused on representation analysis, but recent work focused on analyzing neurons within these models. |
| Approach: | They propose to analyze neural networks to uncover linguistic concepts captured by the network . they propose to use a granular approach to analyze neurons within these models . |
| Outcome: | The proposed method combines methods to discover and understand neurons in a network with evaluation methods. |
Evaluating Saliency Methods for Neural Language Models (2021.naacl-main)
Copied to clipboard
| Challenge: | a general complaint of neural network models is that their internal decision mechanisms are hard to understand. |
| Approach: | They evaluate the quality of prediction interpretations from two perspectives: plausibility and faithfulness. |
| Outcome: | The evaluation of saliency methods on neural language models shows they can be trusted . the methods can be used to interpret the same prediction, but they disagree on interpretations . |
Evaluating Readability and Faithfulness of Concept-based Explanations (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for evaluating concepts from different perspectives lack a unified formalization. |
| Approach: | They propose a formal definition of concepts generalizing to diverse concept-based explanations’ settings and apply it to other types of explanations or tasks. |
| Outcome: | Extensive experimental analysis was carried out to determine the evaluation measures for explanation evaluation measures. |
A Causal Lens for Evaluating Faithfulness Metrics (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) offer natural language explanations as an alternative to feature attribution methods for model interpretability, but they may not reflect the model’s truereasoning faithfully. |
| Approach: | They propose a testbed framework for evaluating faithfulness metrics for natural language explanations using diagnosticity and model-editing methods. |
| Outcome: | The proposed framework evaluates faithfulness metrics for natural language explanations on four tasks including fact-checking, analogy, object counting, and multi-hop reasoning. |
SHAP-Based Explanation Methods: A Review for NLP Interpretability (2022.coling-1)
Copied to clipboard
| Challenge: | Existing models with opacity problems have been proposed to address this problem. |
| Approach: | They propose a unified local-interpretability framework with a rigorous theoretical foundation on the game-theoretic concept of Shapley values. |
| Outcome: | The proposed framework is based on the Shapley-value-based model explanations. |
Faithful Serum: Mitigating the Faithfulness Gap in Textual Explanations of LLM Decisions via Attribution Guidance (2026.acl-long)
Copied to clipboard
| Challenge: | Prior work has focused on generating convincing rationales that appear to be subjectively faithful, but it remains unclear whether these explanations are epistemic faithful. |
| Approach: | They propose a method that enhances epistemic faithfulness by guiding explanation generation through attention-level interventions, informed by token-level heatmaps. |
| Outcome: | The proposed method significantly improves epistemic faithfulness across multiple models, benchmarks, and prompts. |