NILE : Natural Language Inference with Faithful Natural Language Explanations (2020.acl-main)
Copied to clipboard
| Challenge: | Recent growth in popularity of deep learning models on NLP classification tasks has accompanied the need for generating some form of natural language explanation of predicted labels. |
| Approach: | They propose a novel method which generates labels along with its faithful explanations. |
| Outcome: | The proposed method is more accurate than previously reported methods and has higher sensitivity than previous methods. |
Similar Papers
Faithfulness Tests for Natural Language Explanations (2023.acl-short)
Copied to clipboard
Pepa Atanasova, Oana-Maria Camburu, Christina Lioma, Thomas Lukasiewicz, Jakob Grue Simonsen, Isabelle Augenstein
| Challenge: | Existing methods for explaining neural models are misleading as they often present reasons that are unfaithful to the model’s inner workings. |
| Approach: | They propose a counterfactual input editor for inserting reasons that lead to counterfacts but are not reflected by the NLEs. |
| Outcome: | The proposed model can evaluate emerging NLE models, proving a fundamental tool in the development of faithful explanations. |
On Sample Based Explanation Methods for NLP: Faithfulness, Efficiency and Semantic Evaluation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for explaining "black-box" models such as Influence Functions are becoming more popular. |
| Approach: | They propose a semantic-based evaluation metric that can better align with humans’ judgment of explanations than the widely adopted diagnostic or re-training measures. |
| Outcome: | The proposed method can better align with humans’ judgment of explanations than diagnostic or re-training measures. |
Atomic Inference for NLI with Generated Facts as Atoms (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing models that can provide accurate explanations are not interpretable, i.e. they do not reflect the inner workings of the model. |
| Approach: | They propose to use LLM-generated facts as atoms to make interpretable models that can be used to make accurate predictions for each component part of an input. |
| Outcome: | The proposed method outperforms existing methods on natural language understanding tasks with a multi-stage fact generation process and a training regime that incorporates the facts. |
FaithLM: Towards Faithful Explanations for Large Language Models (2026.eacl-long)
Copied to clipboard
Yu-Neng Chuang, Guanchu Wang, Chia-Yuan Chang, Ruixiang Tang, Shaochen Zhong, Fan Yang, Andrew Wen, Mengnan Du, Xuanting Cai, Vladimir Braverman, Xia Hu
| Challenge: | Large language models (LLMs) produce natural language explanations, but they lack faithfulness and do not reflect the evidence the model uses to decide. |
| Approach: | They propose a model-agnostic framework that evaluates and improves the faithfulness of LLM explanations without token masking or task-specific heuristics. |
| Outcome: | The proposed framework improves faithfulness of large language models without masking or heuristics. |
Faithful and Plausible Natural Language Explanations for Image Classification: A Pipeline Approach (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing explanation methods for image classification struggle to provide faithful and plausible explanations for predictions. |
| Approach: | They propose a natural language explanation method that can be applied to any CNN-based classifier without altering its training process or affecting predictive performance. |
| Outcome: | The proposed method can be applied to any CNN-based classifier without altering its training process or affecting predictive performance. |
Stress Test Evaluation for Natural Language Inference (C18-1)
Copied to clipboard
| Challenge: | Existing models perform well at standard datasets for NLI, achieving impressive results across different genres of text. |
| Approach: | They propose to use automatic stress tests to evaluate models' ability to make inferential decisions. |
| Outcome: | The proposed model performs well across genres of text, but lacks the ability to make inferential decisions. |
Leakage-Adjusted Simulatability: Can Models Generate Non-Trivial Explanations of Their Behavior in Natural Language? (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models that generate NL explanations for tasks have been evaluated on the basis of surface-level similarities to human explanations, both through automatic metrics like BLEU and human evaluations. |
| Approach: | They propose to use a model as a proxy for a human observer to evaluate NL explanations from the model simulatability perspective. |
| Outcome: | The proposed model-generated explanations are evaluated on the basis of surface-level similarities to human explanations, both through automatic metrics like BLEU and human evaluations. |
Towards Faithful Natural Language Explanations: A Study Using Activation Patching in Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are capable of generating persuasive Natural Language Explanations (NLEs) however, the faithfulness of these explanations should not be readily trusted at face value. |
| Approach: | They propose to use a causal mediation technique called activation patching to measure the faithfulness of an explanation towards supporting the explained answer. |
| Outcome: | The proposed metric, Causal Faithfulness, quantifies the consistency of causal attributions between explanations and the corresponding model outputs as the indicator of faithfulness. |
Generating Token-Level Explanations for Natural Language Inference (N19-1)
Copied to clipboard
| Challenge: | Existing methods to generate token-level explanations for NLI on single sentences have not been tested. |
| Approach: | They propose to generate token-level explanations for NLI without explicitly annotating training data. |
| Outcome: | The proposed approach is faster and more accurate than the black-box methods. |
A Study of Automatic Metrics for the Evaluation of Natural Language Explanations (2021.eacl-main)
Copied to clipboard
| Challenge: | a lack of transparency is a key issue for robotics and AI. |
| Approach: | They propose to map existing automatic evaluation methods for natural language generation onto explanations. |
| Outcome: | The proposed model shows that embedding-based evaluation methods have higher correlations with human ratings than word-overlap metrics. |