Milad Alshomary, Felix Lange, Meisam Booshehri, Meghdut Sengupta, Philipp Cimiano, Henning Wachsmuth
| Challenge: | Existing studies have focused on the interaction of explanation moves, dialogue acts, and topics in successful dialogues with expert explainers. |
| Approach: | They construct a corpus of 399 reddit dialogues and analyze interaction flows and explainee quality using two language models that can handle long inputs. |
| Outcome: | The proposed model predicts that the interaction flows between the explainer and the explainee correlate with the quality of the explanations in terms of a successful understanding on the explain's side. |
Similar Papers
“Mama Always Had a Way of Explaining Things So I Could Understand”: A Dialogue Corpus for Learning to Construct Explanations (2022.coling-1)
Copied to clipboard
| Challenge: | a new corpus of dialogical explanations is created to help explainable AI . a linguistic analysis of 65 transcribed English dialogues shows that explanations are co-constructed in a dialogue between the explainer and the explainee . |
| Approach: | They propose a corpus of dialogical explanations that are co-constructed in a dialogue . they analyze linguistic patterns of explainers and explainees and explore differences . |
| Outcome: | The proposed corpus of dialogical explanations enables NLP research on how humans explain . the analysis shows that sequence information helps predicting topics, acts, and moves effectively . |
InterroLang: Exploring NLP Models and Datasets through Dialogue-based Explanations (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work on NLP explainability methods lacks a dialogue-based interpretability framework that can convey faithful explanations in human-understandable terms. |
| Approach: | They adapt the conversational explanation framework TalkToModel to the NLP domain and add new NLP-specific operations such as free-text rationalization to illustrate its generalizability. |
| Outcome: | The proposed framework can be used to explain models on three NLP tasks and is generalizable to different datasets, use cases and models. |
Do Explanations Help Users Detect Errors in Open-Domain QA? An Evaluation of Spoken vs. Visual Explanations (2021.findings-acl)
Copied to clipboard
| Challenge: | despite interest in explainable AI, there is increasing skepticism as to whether explanations are useful to end-users in downstream applications. |
| Approach: | They conduct user studies to measure whether explanations help users decide when to accept or reject an ODQA system's answer. |
| Outcome: | The proposed study shows that explanations outperform baselines across modalities but the best strategy varies with the modality. |
QED: A Framework and Dataset for Explanations in Question Answering (2021.tacl-1)
Copied to clipboard
Matthew Lamm, Jennimaria Palomaki, Chris Alberti, Daniel Andor, Eunsol Choi, Livio Baldini Soares, Michael Collins
| Challenge: | Existing question answering systems provide no explanation of reasoning that leads to answer . linguistically informed, extensible framework provides explanations in question answering . |
| Approach: | They propose a linguistically informed, extensible framework for explanations in question answering . they propose an expert-annotated dataset of QED explanations built upon a subset of the Natural Questions dataset . |
| Outcome: | The proposed framework improves the ability of untrained raters to spot errors in QA datasets. |
A Survey of the State of Explainable AI for Natural Language Processing (2020.aacl-main)
Copied to clipboard
| Challenge: | Recent years have seen significant advances in the quality of state-of-the-art models, but they have come at the expense of models becoming less interpretable. |
| Approach: | This survey examines the current state of Explainable AI within the domain of NLP . they detail the operations and explainability techniques currently available for generating explanations for NLP models . |
| Outcome: | This survey examines the state of explainable AI (XAI) within the domain of natural language processing . it focuses on the operations and explainability techniques currently available for NLP models . |
Evaluating Input Feature Explanations through a Unified Diagnostic Evaluation Framework (2025.naacl-long)
Copied to clipboard
| Challenge: | Input feature explanations reveal how a model makes decisions based on a specific input. |
| Approach: | They propose a framework that facilitates an automated comparison between highlight and interactive explanations comprised of four diagnostic properties. |
| Outcome: | The proposed framework compares highlight and interactive explanations across two datasets and two models and shows that interactive span explanations outperform other explanation types across most diagnostic properties. |
On the Diversity and Limits of Human Explanations (2022.naacl-main)
Copied to clipboard
| Challenge: | a growing effort in NLP aims to build datasets of human explanations, but it remains unclear whether they serve their intended goals. |
| Approach: | They argue that the term "explanation" is overloaded and refers to a broad range of notions with different properties and ramifications. |
| Outcome: | The proposed datasets examine the diversity of explanations and their use in NLP. |
FlowDelta: Modeling Flow Information Gain in Reasoning for Conversational Machine Comprehension (D19-58)
Copied to clipboard
| Challenge: | Existing machine comprehension models focus on a single-turn setting and do not account for previous reasoning processes. |
| Approach: | They propose to explicitly model the information gain through the dialogue reasoning . they propose to apply the proposed mechanism to other machine comprehension models . |
| Outcome: | The proposed model achieves state-of-the-art performance in a conversational QA dataset QuAC and a sequential instruction understanding dataset SCONE. |
Learning to Explain: Datasets and Models for Identifying Valid Reasoning Chains in Multihop Question-Answering (2020.emnlp-main)
Copied to clipboard
| Challenge: | despite rapid progress in multihop question-answering, models still have trouble explaining why an answer is correct. |
| Approach: | They propose three explanation datasets in which explanations from corpus facts are annotated . they first annotate multiple candidate explanations for each answer, then use crowd-sourcing perturbations to test generalization . |
| Outcome: | The proposed datasets improve explanation quality but still behind the upper bound . the proposed dataset can be used to improve explanations using a BERT-based classifier . |
On Evaluating Explanation Utility for Human-AI Decision Making in NLP (2024.findings-emnlp)
Copied to clipboard
| Challenge: | a lack of evidence that explanations help people in situations they are introduced for is a problem in NLP . prior work on explainability has focused on overcoming technical challenges and used proxy evaluations. |
| Approach: | They propose to use existing metrics to evaluate the effectiveness of explanations in NLP . they argue that providing AI predictions does not cause decision makers to speed up work . |
| Outcome: | The proposed evaluations show that providing AI predictions does not cause decision makers to speed up their work without compromising performance. |