Identifying Semantic Induction Heads to Understand In-Context Learning (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated remarkable performance, but lack of transparency in their inference logic raises concerns about their trustworthiness. |
| Approach: | They conduct a detailed analysis of the operations of attention heads to understand their in-context learning of LLMs. |
| Outcome: | The proposed analysis of attention heads reveals that they increase the output logits of object tokens and recall objects . the proposed model is a novel approach to understand the in-context learning of large language models. |
Similar Papers
Interpreting Context Look-ups in Transformers: Investigating Attention-MLP Interactions (2024.emnlp-main)
Copied to clipboard
| Challenge: | Using a method to identify next-token neurons, we find that some attention heads recognize contexts relevant to predicting a token and activate a downstream token-predicting neuron accordingly. |
| Approach: | They propose a method to identify next-token neurons and determine the upstream attention heads responsible for their activity in LLMs. |
| Outcome: | The proposed method identifies next-token neurons, finds prompts that highly activate them, and determines the upstream attention heads responsible. |
Analyzing the Attention Heads for Pronoun Disambiguation in Context-aware Machine Translation Models (2025.coling-main)
Copied to clipboard
| Challenge: | In Context-aware Machine Translation, the context sentences are available to the system and can be used to maintain coherence of translation and resolve ambiguities. |
| Approach: | They investigate the role of attention heads in Context-aware Machine Translation models for pronoun disambiguation in the English-to-German and English- to-French directions. |
| Outcome: | The attention heads influence the models' ability to disambiguate pronouns in the English-to-German and English- to-French directions. |
Exploring Attention Attractors in Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies have suggested that attention attractors function as "summary tokens" while others speculate that tokens with weaker semantics attract high attention, they act as attention sinks that offload excessive attention. |
| Approach: | They examine attention attractors, tokens that draw significantly high attention, in large language models. |
| Outcome: | The proposed models are able to capture long-range dependencies within a given context. |
Plausibility Processing in Transformer Language Models: Focusing on the Role of Attention Heads in GPT (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Using attention heads, we can explore how Transformer language models process semantic knowledge, especially regarding the plausibility of noun-verb relations. |
| Approach: | They propose to investigate how Transformer language models process semantic knowledge, especially regarding the plausibility of noun-verb relations. |
| Outcome: | The proposed model exhibits a higher degree of similarity with humans in plausibility processing compared to other Transformer language models. |
Out-of-Context Reasoning in Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a lightweight technique trains only new token embeddings on axioms and evaluates them on unseen tasks. |
| Approach: | They propose a lightweight technique that trains only new token embeddings on axioms . they train only new embeddables and evaluate them on unseen tasks . |
| Outcome: | The proposed technique trains only new token embeddings on axioms and evaluates them on unseen tasks. |
A Critical Study of What Code-LLMs (Do Not) Learn (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models trained on code corpora have limitations such as suggesting codes with syntactic errors, variable misuse etc. |
| Approach: | They conduct a fine-grained analysis of attention maps and hidden representations of large-scale Large Language Models (cLLMs) trained on a large corpus of code and natural language -programming language pairs. |
| Outcome: | The proposed models encode relations among syntactic tokens and identifiers, but fail to encode relations between syntaktic token and identifier. |
Attention Can Reflect Syntactic Structure (If You Let It) (2021.eacl-main)
Copied to clipboard
| Challenge: | a recent study has attempted to decode linguistic structure from the Transformer . but, much of the work focused on English, a language with rigid word order and a lack of inflectional morphology. |
| Approach: | They propose to fine-tune a feature encoder for BERT to learn linguistic structure from its multi-head attention mechanism. |
| Outcome: | The proposed model can decode full trees above baseline accuracy from single attention heads across languages. |
How Do Multilingual Language Models Remember Facts? (2025.findings-acl)
Copied to clipboard
| Challenge: | Prior research has focused on English monolingual models, but how these mechanisms generalize to non-English languages remains unexplored. |
| Approach: | They analyze three multilingual LLMs to find out how they can generalize recall mechanisms . they find that subject enrichment is language-independent, object extraction is language dependent . |
| Outcome: | The proposed model performs better in multilingual contexts than in English models . the model is more efficient in multi-lingual context, but it is more complex in multilinguistic models compared to English models. |
How do Large Language Models Learn In-Context? Query and Key Matrices of In-Context Heads are Two Towers for Metric Learning (2024.emnlp-main)
Copied to clipboard
| Challenge: | In-context learning (ICL) is an emergent ability of large language models. |
| Approach: | They propose to use in-context learning to predict sentences with semantically-unrelated labels on 1% heads to investigate the mechanism. |
| Outcome: | The proposed methods reduce the majority label bias and recency bias by 22% and 17%, respectively. |
Induction Heads as an Essential Mechanism for Pattern Matching in In-context Learning (2025.findings-naacl)
Copied to clipboard
| Challenge: | Large language models have shown a remarkable ability to learn and perform complex tasks through in-context learning (ICL). |
| Approach: | They analyse two state-of-the-art models, Llama-3-8B and InternLM2-20B on abstract pattern recognition and NLP tasks. |
| Outcome: | The proposed model can perform up to 32% better than previous models on abstract pattern recognition and NLP tasks. |