Explaining Black Box Predictions and Unveiling Data Artifacts through Influence Functions (2020.acl-main)
Copied to clipboard
| Challenge: | Modern deep learning models for NLP are notoriously opaque, and this has motivated efforts to design example-specific approaches to interpret such models. |
| Approach: | They propose to use influence functions to explain models by highlighting important words in input text to provide models with an explanation. |
| Outcome: | The proposed approach is particularly useful for natural language inference, a task in which ‘saliency maps’ may not have clear interpretation. |
Similar Papers
Interpreting Predictions of NLP Models (2020.emnlp-tutorials)
Copied to clipboard
| Challenge: | This tutorial will provide a background on interpretation techniques for neural NLP models. |
| Approach: | This tutorial will provide a background on interpretation techniques for NLP models . it will examine saliency maps, input perturbations, adversarial attacks and influence functions . |
| Outcome: | This tutorial will provide a background on interpretation techniques . examples-specific interpretations include saliency maps, input perturbations, adversarial attacks, influence functions . |
Do Influence Functions Work on Large Language Models? (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Influence functions are important for quantifying the impact of individual training data points on a model’s predictions. |
| Approach: | They conduct a systematic study to address a key question: do influence functions work on large language models? |
| Outcome: | The influence functions perform poorly across multiple tasks and are therefore unsuitable for large language models. |
Influence Tuning: Demoting Spurious Correlations via Instance Attribution and Instance-Driven Updates (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to interpret black-box models to learn spurious correlations are not well understood. |
| Approach: | They propose a procedure that leverages model interpretations to update parameters towards a plausible interpretation rather than an interpretation that relies on spurious patterns in data. |
| Outcome: | The proposed procedure outperforms baseline methods that use adversarial training in a controlled setup. |
Make Every Example Count: On the Stability and Utility of Self-Influence for Learning from Noisy NLP Datasets (2023.emnlp-main)
Copied to clipboard
| Challenge: | Increasingly larger datasets have become a standard ingredient to advancing the state-of-the-art in NLP, however, data quality might have already become the bottleneck to unlock further gains. |
| Approach: | They propose a general method for improving model performance in the presence of noisy training data based on self-influence and bandit curriculum learning. |
| Outcome: | The proposed method improves model performance in machine translation, question answering and text classification, building up on approaches to self-influence calculation and automated curriculum learning. |
Influence Functions for Sequence Tagging Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Named Entity Recognition, Part-of-Speech tagging, and Semantic Role Labeling are standard tasks in NLP, but there has been little work on interpretability methods for sequence taging. |
| Approach: | They propose to extend influence functions to sequence tagging tasks by identifying noisy annotations in NER corpora. |
| Outcome: | The proposed methods are able to identify noisy annotations in NER corpora and are scalable. |
InfFeed: Influence Functions as a Feedback to Improve the Performance of Subjective Tasks (2024.lrec-main)
Copied to clipboard
| Challenge: | InfFeed uses influence functions to compute the influential instances for a target instance. |
| Approach: | They propose an apparatus that uses influence functions to compute the influential instances for a target instance. |
| Outcome: | The proposed model outperforms the state-of-the-art baselines by 4% for hate speech classification, 3.5% for stance classification, and 3% for irony and 2% for sarcasm detection. |
Interpreting Recurrent and Attention-Based Neural Models: a Case Study on Natural Language Inference (D18-1)
Copied to clipboard
| Challenge: | In this paper, we examine the behavior of deep learning models in their intermediate layers . saliency determines what is critical for the final decision of a deep model . |
| Approach: | They propose to interpret the intermediate layers of deep models by visualizing the saliency of attention and LSTM gating signals. |
| Outcome: | The proposed methods reveal interesting insights and identify critical information contributing to the model decisions. |
An Empirical Comparison of Instance Attribution Methods for NLP (2021.naacl-main)
Copied to clipboard
| Challenge: | Influence functions provide machinery for identifying training instances that may have led to a specific prediction, but are computationally expensive and prohibitive in many cases. |
| Approach: | They evaluate the degree to which different potential instance attribution agrees with respect to the importance of training samples. |
| Outcome: | The proposed methods exhibit desirable characteristics similar to more complex methods, but are computationally expensive. |
Interpretation of NLP models through input marginalization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to interpret NLP predictions replace each token with a predefined value, resulting in misleading interpretations. |
| Approach: | They propose to marginalize each token out of the training data distribution to demystify the "black box" property of deep neural networks for natural language processing. |
| Outcome: | The proposed method marginalizes each token out of the training data distribution. |
Influence Scores at Scale for Efficient Language Data Sampling (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have shown that ML models can be fine-tuned on as much data as possible without degradation in performance metrics. |
| Approach: | They evaluate the applicability of influence scores in language classification tasks by random sampling and stress-testing one of the scores. |
| Outcome: | The proposed model can be fine-tuned on 50% of the original data without degradation in performance metrics. |