Reversed Attention: On The Gradient Descent Of Attention Layers In GPT (2025.naacl-long)
Copied to clipboard
| Challenge: | In this work, we examine the attention maps obtained from the backward pass of attention, which we call "Reversed Attention" (RA). |
| Approach: | They propose to use a method called "attention patching" to alter the forward pass of attention without modifying the model's weights. |
| Outcome: | The proposed method enables the model to alter the forward pass of attention without altering the model’s weights. |
Similar Papers
Backward Lens: Projecting Language Model Gradients into the Vocabulary Space (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent interpretability methods project weights and hidden states obtained from the forward pass to the models’ vocabularies, helping to uncover how information flows within LMs. |
| Approach: | They propose to cast a gradient matrix as a low-rank linear combination of forward and backward passes’ inputs and then to project these gradients into vocabulary items. |
| Outcome: | The proposed method can be cast as a low-rank linear combination of forward and backward passes’ inputs and project these gradients into vocabulary items. |
VISIT: Visualizing and Interpreting the Semantic Information Flow of Transformers (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work in interpretability suggests we can project weights and hidden states of transformer-based language models (LMs) to their vocabulary space, a transformation that makes them more human interpretable. |
| Approach: | They propose a tool to visualize a forward pass of Generative Pre-trained Transformers as an interactive flow graph with nodes representing neurons or hidden states and edges representing interactions between them. |
| Outcome: | The proposed visualization simplifies huge amounts of data into easy-to-read graphs that can reflect the models’ internal processing, uncovering the contribution of each component to the models' final prediction. |
Incorporating Residual and Normalization Layers into Analysis of Masked Language Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Transformer architecture is composed of multi-head attention, which has been extensively analyzed. |
| Approach: | They extended the scope of the analysis of Transformers from solely the attention patterns to the whole attention block, i.e., multi-head attention, residual connection, and layer normalization. |
| Outcome: | The proposed method incorporates the whole attention block, i.e., multi-head attention, residual connection, and layer normalization into the analysis. |
Attention Calibration for Transformer in Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Attention mechanisms have been ubiquitous in neural machine translation (NMT) however, many studies doubt whether highlyattended inputs have a large impact on the model outputs. |
| Approach: | They propose to introduce a mask perturbation model that automatically evaluates each input’s contribution to the model outputs. |
| Outcome: | The proposed model is more uniform at lower layers while more concentrated on the specific inputs at higher layers. |
Cross-Attention is All You Need: Adapting Pretrained Transformers for Machine Translation (2021.emnlp-main)
Copied to clipboard
| Challenge: | a series of experiments show that fine-tuning only the cross-attention parameters is nearly as effective as fine-timing all parameters. |
| Approach: | They conduct experiments to fine-tune a translation model on data where either the source or target language has changed. |
| Outcome: | The proposed model can be trained to several new languages with reduced parameter storage overhead. |
Mask Attention Networks: Rethinking and Strengthen Transformer (2021.naacl-main)
Copied to clipboard
Zhihao Fan, Yeyun Gong, Dayiheng Liu, Zhongyu Wei, Siyuan Wang, Jian Jiao, Nan Duan, Ruofei Zhang, Xuanjing Huang
| Challenge: | Existing research explores to enhance the two sublayers separately to improve the capability of Transformer for text representation. |
| Approach: | They propose to combine SAN and Feed-Forward Networks to create a dynamic mask attention network with a learnable mask matrix which can model localness adaptively. |
| Outcome: | The proposed model outperforms the original Transformer on translation and text summarization tasks. |
RealFormer: Transformer Likes Residual Attention (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing techniques to create Residual Attention Layer Transformer networks outperform the canonical Transformer on a wide spectrum of tasks. |
| Approach: | They propose a technique to create Residual Attention Layer Transformer networks that outperform the canonical Transformer on a wide spectrum of tasks. |
| Outcome: | The proposed technique outperforms the canonical Transformer on a wide spectrum of tasks including Masked Language Modeling, GLUE, SQUAD, Neural Machine Translation, WikiHop, HotpotQA, Natural Questions, and OpenKP. |
Characterizing the Expressivity of Local Attention in Transformers (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies show that global and local attention are expressively complementary. |
| Approach: | They propose to restrict global attention to a fixed-size window of preceding tokens . they also propose to add local attention to local-only transformers to increase model quality . |
| Outcome: | The proposed model outperforms the global–local transformers on formal language recognition and natural language modeling. |
Mitigating Attention Localization in Small Scale: Self-Attention Refinement via One-step Belief Propagation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new framework for self-attention models is proposed to address this problem . it injects *multi-hop* relationships into the attention graph, allowing for better performance . |
| Approach: | They propose a framework that injects *multi-hop* relationships through a belief propagation process. |
| Outcome: | The proposed framework helps prevent entropy collapse in deeper layers and maintains GTD at task-appropriate levels. |
A Multiscale Visualization of Attention in the Transformer Model (P19-3)
Copied to clipboard
| Challenge: | Various tools have been developed to visualize attention in NLP models, ranging from attention-matrix heatmaps to bipartite graph representations. |
| Approach: | They propose an open-source tool that visualizes attention at multiple scales and provides a unique perspective on the attention mechanism. |
| Outcome: | The proposed model outperforms OpenAI GPT-2 and BERT on several language modeling benchmarks. |