On the Interpretability of Deep Learning Models for Collaborative Argumentation Analysis in Classrooms (2024.acl-srw)
Copied to clipboard
| Challenge: | Existing models for collaborative argumentation lack interpretability and teachers are skeptics about their use. |
| Approach: | They propose to use four explainable AI methods to provide models for automated analysis of argument moves and specificity levels within collaborative argumentation to cultivate trust among teachers. |
| Outcome: | The proposed models perform exceptionally well in analyzing word contributions and demonstrating that the models can be explained by a user-interface. |
Similar Papers
Discussion Tracker: Supporting Teacher Learning about Students’ Collaborative Argumentation in High School Classrooms (2020.coling-demos)
Copied to clipboard
| Challenge: | Discussion Tracker provides teachers with data about argument moves, specificity and collaboration . |
| Approach: | They have developed a classroom discussion analytics system that leverages natural language processing to classify argument moves, specificity and collaboration. |
| Outcome: | The proposed system performs with moderate to substantial agreement with humans in a classroom setting. |
On Evaluating Explanation Utility for Human-AI Decision Making in NLP (2024.findings-emnlp)
Copied to clipboard
| Challenge: | a lack of evidence that explanations help people in situations they are introduced for is a problem in NLP . prior work on explainability has focused on overcoming technical challenges and used proxy evaluations. |
| Approach: | They propose to use existing metrics to evaluate the effectiveness of explanations in NLP . they argue that providing AI predictions does not cause decision makers to speed up work . |
| Outcome: | The proposed evaluations show that providing AI predictions does not cause decision makers to speed up their work without compromising performance. |
Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior? (2020.acl-main)
Copied to clipboard
| Challenge: | a new study examines the impact of algorithmic explanations on simulatability of machine learning models . a model is simulatable when a person can predict its behavior on new inputs . |
| Approach: | They conduct human subject tests to isolate effect of algorithmic explanations on simulatability . they find ratings of explanations are not predictive of how helpful they are . |
| Outcome: | The results provide the first reliable estimates of how explanations influence simulatability . they show that ratings are not predictive of how helpful explanations are . |
Investigating the Role of Argumentation in the Rhetorical Analysis of Scientific Publications with Neural Multi-Task Learning Models (D18-1)
Copied to clipboard
| Challenge: | Scientific publications are argumentative and often adhere to well-trodden rhetorical patterns and argumentation schemes. |
| Approach: | They investigate the link between scientific publications and rhetorical aspects such as discourse categories or citation contexts by coupling rhetorical classifiers with extraction of argumentative components. |
| Outcome: | The proposed models show significant performance gains for different rhetorical analysis tasks. |
Unsupervised Argumentation Mining in Student Essays (2020.lrec-1)
Copied to clipboard
| Challenge: | State-of-the-art argumentation mining systems rely on annotated training data and are supervised, thus relying on an annotation of the components and relationships between them. |
| Approach: | They propose to bootstrap from a small set of argument components automatically identified using simple heuristics in combination with reliable contextual cues. |
| Outcome: | The proposed approach outperforms two supervised baselines and achieves 73.5-83.7% of the performance of a state-of-the-art neural approach. |
This Reads Like That: Deep Learning for Interpretable Natural Language Processing (2023.emnlp-main)
Copied to clipboard
| Challenge: | In this work, we explore the extension of prototypical networks to natural language processing. |
| Approach: | They propose a weighted similarity measure that enhances the similarity computation by focusing on informative dimensions of pre-trained sentence embeddings. |
| Outcome: | The proposed method improves predictive performance on AG News and RT Polarity datasets and the rationale-based recurrent convolutions. |
Saliency Learning: Teaching the Model Where to Pay Attention (N19-1)
Copied to clipboard
| Challenge: | Recent work on explanation and interpretation has introduced methods to provide insights toward the model’s behaviour and predictions, but they do not improve the model's reliability. |
| Approach: | They propose to provide explanation training and ensure alignment of model’s explanation with ground truth explanation to ensure the model makes correct predictions for the right reason. |
| Outcome: | The proposed method produces more reliable predictions while delivering better results compared to traditional models. |
Human-grounded Evaluations of Explanation Methods for Text Classification (D19-1)
Copied to clipboard
| Challenge: | Explainable Artificial Intelligence (XAI) is aimed at providing explanations for decisions made by AI systems. |
| Approach: | They propose to use model-agnostic and model-specific explanation methods for CNNs for text classification to provide human-grounded evaluations. |
| Outcome: | The proposed methods could be used to explain models' results and improve AIs and humans in many cases. |
Argument Quality Assessment in the Age of Instruction-Following Large Language Models (2024.lrec-main)
Copied to clipboard
Henning Wachsmuth, Gabriella Lapesa, Elena Cabrio, Anne Lauscher, Joonsuk Park, Eva Maria Vecchi, Serena Villata, Timon Ziegenbein
| Challenge: | Argument quality assessment is critical for opinion formation, decision making, writing education, and the like. |
| Approach: | They propose to use large language models to leverage knowledge across contexts to enable a much more reliable assessment. |
| Outcome: | The proposed approach improves the quality of argumentation and the ability to leverage knowledge across contexts. |
Limited Generalizability in Argument Mining: State-Of-The-Art Models Learn Datasets, Not Arguments (2025.acl-long)
Copied to clipboard
| Challenge: | Identifying arguments is a prerequisite for various tasks in automated discourse analysis. |
| Approach: | They evaluate four BERT-like transformers on 17 English sentence-level datasets . they find that they tend to rely on lexical shortcuts tied to content words . |
| Outcome: | The proposed models perform best on 17 English sentence-level datasets on common tasks, but their performance drops when applied to unseen datasets. |