Attention weights accurately predict language representations in the brain (2022.findings-emnlp)
Copied to clipboard
| Challenge: | In Transformer-based language models, the attention mechanism converts token embeddings into contextual embeddables that incorporate information from neighboring words. |
| Approach: | They analyze fMRI recordings of English language learners and extract attention weights from them to determine how well they can predict brain responses. |
| Outcome: | The resulting hidden state embeddings are more accurate than lexical embeddngs or RNN-based models. |
Similar Papers
Attention Weights in Transformer NMT Fail Aligning Words Between Sequences but Largely Explain Model Predictions (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Using attention weights, we show that NMT models make alignment errors by relying on uninformative tokens from the source sequence. |
| Approach: | They propose to use attention weights to regulate alignment errors in NMT models . they propose methods that largely reduce the word alignment error rate compared to standard induced alignments from attention weighted tokens. |
| Outcome: | The proposed methods reduce the word alignment error rate compared to standard induced alignments from attention weights. |
Neural Language Taskonomy: Which NLP Tasks are the most Predictive of fMRI Brain Activity? (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing literature has focused on pretrainer-based text-driven brain encoding models . however, few studies have explored the efficacy of task-specific learning of Transformers . |
| Approach: | They propose to use ten popular natural language processing tasks to learn Transformer representations for predicting brain responses. |
| Outcome: | The proposed model predicts brain activity across the whole brain. |
Do Multilingual Neural Machine Translation Models Contain Language Pair Specific Attention Heads? (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent studies on multilingual representations focus on whether there is an emergence of language-independent representations or whether multilingual models partition their weights among different languages. |
| Approach: | They analyze encoder self-attention and encoder-decoder attention heads in a multilingual neural translation model. |
| Outcome: | The proposed model is based on a multilingual neural translation model with a language-independent representation. |
Transformers Can Represent n-gram Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing work studies the transformer architecture in terms of language acceptance . n-gram LMs can represent any ngram lm, giving a concrete lower bound on their probabilistic representational capacity. |
| Approach: | They propose to use transformer LMs to represent n-gram LM, a class of language models . they show transformer lms can exactly represent any ngram lmm, giving them a lower bound . |
| Outcome: | The proposed model can represent any n-gram LM, giving a lower bound on their probabilistic representational capacity. |
Identifying Semantic Induction Heads to Understand In-Context Learning (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated remarkable performance, but lack of transparency in their inference logic raises concerns about their trustworthiness. |
| Approach: | They conduct a detailed analysis of the operations of attention heads to understand their in-context learning of LLMs. |
| Outcome: | The proposed analysis of attention heads reveals that they increase the output logits of object tokens and recall objects . the proposed model is a novel approach to understand the in-context learning of large language models. |
Linking artificial and human neural representations of language (D19-1)
Copied to clipboard
| Challenge: | a pre-trained BERT architecture is used to fine-tune sentence encoding models on a variety of natural language understanding (NLU) tasks. |
| Approach: | They compare sentence encoding models with fMRI-based fMR predictions of the sentence . they use a pre-trained BERT architecture as a baseline and fine-tune it on a variety of natural language understanding (NLU) tasks. |
| Outcome: | The proposed model does not yield significant improvements in brain decoding performance on the natural language understanding (NLU) tasks. |
Interpreting Context Look-ups in Transformers: Investigating Attention-MLP Interactions (2024.emnlp-main)
Copied to clipboard
| Challenge: | Using a method to identify next-token neurons, we find that some attention heads recognize contexts relevant to predicting a token and activate a downstream token-predicting neuron accordingly. |
| Approach: | They propose a method to identify next-token neurons and determine the upstream attention heads responsible for their activity in LLMs. |
| Outcome: | The proposed method identifies next-token neurons, finds prompts that highly activate them, and determines the upstream attention heads responsible. |
Exploring Attention Attractors in Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies have suggested that attention attractors function as "summary tokens" while others speculate that tokens with weaker semantics attract high attention, they act as attention sinks that offload excessive attention. |
| Approach: | They examine attention attractors, tokens that draw significantly high attention, in large language models. |
| Outcome: | The proposed models are able to capture long-range dependencies within a given context. |
VISIT: Visualizing and Interpreting the Semantic Information Flow of Transformers (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work in interpretability suggests we can project weights and hidden states of transformer-based language models (LMs) to their vocabulary space, a transformation that makes them more human interpretable. |
| Approach: | They propose a tool to visualize a forward pass of Generative Pre-trained Transformers as an interactive flow graph with nodes representing neurons or hidden states and edges representing interactions between them. |
| Outcome: | The proposed visualization simplifies huge amounts of data into easy-to-read graphs that can reflect the models’ internal processing, uncovering the contribution of each component to the models' final prediction. |
Attention Can Reflect Syntactic Structure (If You Let It) (2021.eacl-main)
Copied to clipboard
| Challenge: | a recent study has attempted to decode linguistic structure from the Transformer . but, much of the work focused on English, a language with rigid word order and a lack of inflectional morphology. |
| Approach: | They propose to fine-tune a feature encoder for BERT to learn linguistic structure from its multi-head attention mechanism. |
| Outcome: | The proposed model can decode full trees above baseline accuracy from single attention heads across languages. |