Papers by Shahar Katz
Safeguarding Language Models via Self-Destruct Trapdoor (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing mechanisms to restrict behavior of language models (LMs) are vulnerable to misuse and misalignment. |
| Approach: | They propose a mechanism to restrict specific behaviors in language models by exploiting hardware properties. |
| Outcome: | The proposed mechanism can be applied to trigger overflows for specific behaviors or target hardware malfunctions. |
VISIT: Visualizing and Interpreting the Semantic Information Flow of Transformers (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work in interpretability suggests we can project weights and hidden states of transformer-based language models (LMs) to their vocabulary space, a transformation that makes them more human interpretable. |
| Approach: | They propose a tool to visualize a forward pass of Generative Pre-trained Transformers as an interactive flow graph with nodes representing neurons or hidden states and edges representing interactions between them. |
| Outcome: | The proposed visualization simplifies huge amounts of data into easy-to-read graphs that can reflect the models’ internal processing, uncovering the contribution of each component to the models' final prediction. |
TensorLens: End-to-End Transformer Analysis via High-Order Attention Tensors (2026.acl-long)
Copied to clipboard
| Challenge: | Existing attention-aggregation methods focus on individual attention heads or layers, failing to account for the model’s global behavior. |
| Approach: | They propose a unified attention representation that captures the entire transformer as a single, input-dependent linear operator expressed through a high-order attention-interaction tensor. |
| Outcome: | The proposed model encapsulates the entire transformer as a single, input-dependent linear operator expressed through a high-order attention-interaction tensor. |
Backward Lens: Projecting Language Model Gradients into the Vocabulary Space (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent interpretability methods project weights and hidden states obtained from the forward pass to the models’ vocabularies, helping to uncover how information flows within LMs. |
| Approach: | They propose to cast a gradient matrix as a low-rank linear combination of forward and backward passes’ inputs and then to project these gradients into vocabulary items. |
| Outcome: | The proposed method can be cast as a low-rank linear combination of forward and backward passes’ inputs and project these gradients into vocabulary items. |
Reversed Attention: On The Gradient Descent Of Attention Layers In GPT (2025.naacl-long)
Copied to clipboard
| Challenge: | In this work, we examine the attention maps obtained from the backward pass of attention, which we call "Reversed Attention" (RA). |
| Approach: | They propose to use a method called "attention patching" to alter the forward pass of attention without modifying the model's weights. |
| Outcome: | The proposed method enables the model to alter the forward pass of attention without altering the model’s weights. |
Segment-Based Attention Masking for GPTs (2025.acl-long)
Copied to clipboard
| Challenge: | Causal masking is a fundamental component in Generative Pre-Trained Transformers (GPT) models, playing a crucial role during training. |
| Approach: | They propose to apply causal masking to all input tokens step-by-step, mimicking the generation process. |
| Outcome: | The proposed model can process the entire user prompt at once, but it is applied to all input tokens step-by-step, mimicking the generation process. |