Circuit Compositions: Exploring Modular Structures in Transformer-Based Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Recent advances in mechanistic interpretability have made progress in identifying circuits, the minimal computational subgraphs responsible for a model’s behavior on specific tasks. |
| Approach: | They propose to analyze circuits for highly compositional subtasks within a transformer-based language model to determine their modularity and how they relate to each other. |
| Outcome: | The proposed approach shows that the circuits identified exhibit notable node overlap and cross-task faithfulness. |
Similar Papers
Towards Interpretable Sequence Continuation: Analyzing Shared Circuits in Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent work aims to reverse engineer transformer models into human-readable representations . transformers exhibit strong capabilities on linguistic tasks, but their complex architectures make them difficult to interpret. |
| Approach: | They extend transformer models into human-readable representations that implement algorithmic functions by analyzing sequence continuation tasks. |
| Outcome: | The proposed model can be reverse-engineered into human-readable representations that implement algorithmic functions. |
Analyzing the Inner Workings of Transformers in Compositional Generalization (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing studies on compositional generalization abilities of neural models have focused on benchmarks, but the results do not reflect the underlying competence of the model. |
| Approach: | They propose to find an existing subnetwork that contributes to the generalization performance and perform causal analyses on how the model utilizes syntactic features. |
| Outcome: | The proposed model relies on syntactic features but the subnetwork with better generalization performance relies mainly on a non-compositional algorithm . |
Mechanistic Analysis Of Universality: Numerical Comparison Circuits Across Transformer Architectures (2026.acl-srw)
Copied to clipboard
| Challenge: | Mechanistic interpretability seeks to identify internal circuits within transformer language models but it is unclear whether they generalize across model families and scales. |
| Approach: | They propose to identify internal circuits within transformer language models by numerical comparisons. |
| Outcome: | The proposed model implementations are consistent across architecture and scale, the authors show . their results highlight the need for cross model comparisons to claim generalization of internal circuits. |
The Learnability of Model-Theoretic Interpretation Functions in Artificial Neural Networks (2026.findings-acl)
Copied to clipboard
| Challenge: | Entity vectors improve scores on basic event, while gated architectures benefit most. |
| Approach: | They extend entity-level semantic representations, modern architectures, principled competing event generation, extended systematicity tests and a two-dimensional difficulty analysis disaggregating results by modifier complexity. |
| Outcome: | The proposed model-theoretic interpretation functions generalize systematically to out-of-training-sample sentences. |
Mechanistic Unveiling of Transformer Circuits: Self-Influence as a Key to Model Reasoning (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies have shown that large language models implicitly embed reasoning trees, but their internal mechanisms remain largely opaque due to the complexity of non-linear interactions and high-dimensional operations. |
| Approach: | They propose to use circuit analysis and self-influence functions to map the reasoning process of large models. |
| Outcome: | The proposed model is able to map human-interpretable reasoning paths and a model's underlying circuits reveal human-mediated reasoning processes. |
Fine-Grained Analysis of Shared Syntactic Mechanisms in Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Recent advances in language models have demonstrated their ability to process linguistic expressions with complex syntactic structures. |
| Approach: | They investigate whether language models employ shared neural mechanisms across different constructions by applying causal interpretability methods at a granular level. |
| Outcome: | The proposed model performance improves on acceptability judgment benchmarks. |
Emergent Modularity in Pre-trained Transformers (2023.findings-acl)
Copied to clipboard
Zhengyan Zhang, Zhiyuan Zeng, Yankai Lin, Chaojun Xiao, Xiaozhi Wang, Xu Han, Zhiyuan Liu, Ruobing Xie, Maosong Sun, Jie Zhou
| Challenge: | Existing studies on pre-trained Transformers show that they learn fine-grained neuron functions. |
| Approach: | They examine the presence of modularity in pre-trained Transformers . they focus on Mixture-of-Experts, a promising candidate for modularity . |
| Outcome: | The proposed structure stabilizes at the early stage, which is faster than neuron stabilization. |
Transformer-specific Interpretability (2024.eacl-tutorials)
Copied to clipboard
| Challenge: | Transformers are dominant play-ers in various scientific fields, but their inner workings remain opaque. |
| Approach: | This tutorial presents a trending approach to interpreting Transformers . it uses specific features of the Transformer architecture to quantify context- mixing interactions . |
| Outcome: | This tutorial aims to show how a new trending approach can be applied to Transformer-based models. |
How do Transformer Embeddings Represent Compositions? A Functional Analysis (2025.findings-acl)
Copied to clipboard
| Challenge: | Despite the popularity of transformer-based models, little is known about how they represent compound words and whether they are compositional. |
| Approach: | They evaluate compositionality in mistral, OpenAI Large, and Google embedding models and compare them with BERT. |
| Outcome: | The proposed models perform best in addition, multiplication, dilation, regression, and the classic vector addition model performs almost as well as any other model. |
Finding and Editing Multi-Modal Neurons in Pre-Trained Transformers (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to identify key neurons for interpretability of multi-modal large language models are unclear. |
| Approach: | They propose a method to identify key neurons for interpretability by multi-modal large language models. |
| Outcome: | The proposed method improves conventional works upon efficiency and applied range by removing needs of costly gradient computation. |