| Challenge: | Existing studies have focused on whether local attention weights reflect the importance of input representations. |
| Approach: | They propose to analyze for each word token the following two quantities: its polarity score and its attention score, where the latter is a global assessment on the token’s significance. |
| Outcome: | The proposed model can be improved under conditions where the interplay between the two quantities can contribute towards model performance. |
Similar Papers
Is Attention Interpretable? (P19-1)
Copied to clipboard
| Challenge: | Attention mechanisms have recently boosted performance on a range of NLP tasks. |
| Approach: | They propose to manipulate attention weights in text classification models and analyze the resulting differences in their predictions. |
| Outcome: | The proposed approach improves models' predictions by using gradient-based rankings of attention weights. |
Understanding Attention for Vision-and-Language Tasks (2022.coling-1)
Copied to clipboard
| Challenge: | Attention mechanism has been used in Vision-and-Language (VL) tasks to bridge the semantic gap between visual and textual clues. |
| Approach: | They conduct a comprehensive analysis on understanding the role of attention alignment by looking into attention score calculation methods and checking how it represents the visual region’s and textual token’s significance for the global assessment. |
| Outcome: | The attention score calculation methods represent visual region’s and textual token’s significance for the global assessment. |
Rethinking Self-Attention: Towards Interpretability in Neural Parsing (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work shows that attention mechanisms provide arguably explainable attention distributions that can help to interpret predictions. |
| Approach: | They propose a new self-attention layer where attention heads represent labels. |
| Outcome: | The proposed model obtains state-of-the-art results on the Penn Treebank and Chinese Treebank. |
Attention is not Explanation (N19-1)
Copied to clipboard
| Challenge: | Attention mechanisms have seen wide adoption in neural NLP models. |
| Approach: | They perform extensive experiments to assess the degree to which attention weights provide meaningful "explanations" they find that attention weighted inputs are often uncorrelated with gradient-based measures of feature importance . |
| Outcome: | The proposed model is based on a distribution over attended-to input units . the findings show that attention weights are often uncorrelated with features . |
Attention is not not Explanation (D19-1)
Copied to clipboard
| Challenge: | Attention mechanisms play a central role in NLP systems, especially within recurrent neural network (RNN) models. |
| Approach: | They propose to use a simple uniform-weights baseline, a variance calibration and a diagnostic framework to determine when/whether attention can be used as explanation in RNN models. |
| Outcome: | The proposed tests show that even reliable adversarial distributions don't perform well on the simple diagnostic, indicating that prior work does not disprove the usefulness of attention mechanisms for explainability. |
How does Attention Affect the Model? (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on the effectiveness of attention in NLP do not consider changes in semantic capability of different components. |
| Approach: | They propose a framework that exploits a convex hull representation of sequence semantics in an n-dimensional Semantic Euclidean Space and defines indicators to capture the impact of attention on sequence semantic. |
| Outcome: | The proposed framework exploits a convex hull representation of sequence semantics in an n-dimensional Semantic Euclidean Space and defines indicators to capture the impact of attention on sequence semantic. |
Is Attention Explanation? An Introduction to the Debate (2022.acl-long)
Copied to clipboard
Adrien Bibal, Rémi Cardon, David Alfter, Rodrigo Wilkens, Xiaoou Wang, Thomas François, Patrick Watrin
| Challenge: | Attention has been used in various tasks of NLP and other fields of machine learning to increase performance and provide some explanations. |
| Approach: | They propose to use attention as an explanation for deep learning models to increase performance . they propose to apply attention weights to queries and queries based on scalar scores . |
| Outcome: | The proposed model can be used to increase performance while providing some explanations. |
Is Sparse Attention more Interpretable? (2021.acl-short)
Copied to clipboard
| Challenge: | Sparse attention has been claimed to increase model interpretability . however, the attention distribution is typically over representations internal to the model rather than the inputs themselves . |
| Approach: | They conduct experiments to understand how sparsity affects our ability to use attention as an explainability tool. |
| Outcome: | The proposed model does not map to a sparse set of influential inputs, but rather to fewer inputs. |
Exploring Attention Attractors in Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies have suggested that attention attractors function as "summary tokens" while others speculate that tokens with weaker semantics attract high attention, they act as attention sinks that offload excessive attention. |
| Approach: | They examine attention attractors, tokens that draw significantly high attention, in large language models. |
| Outcome: | The proposed models are able to capture long-range dependencies within a given context. |
A Novel Perspective to Look At Attention: Bi-level Attention-based Explainable Topic Modeling for News Classification (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing deep learning models have the attention mechanism to improve performance, but the inherent characteristics of deep learning model complexity and the flexibility of the attention structure make them difficult to explain. |
| Approach: | They propose a two-tier attention architecture to decouple the complexity of explanation and the decision-making process by using large-scale news corpora. |
| Outcome: | The proposed model can achieve competitive performance with state-of-the-art models and illustrates its appropriateness from an explainability perspective. |