Papers by Itamar Zimerman
Faithful Serum: Mitigating the Faithfulness Gap in Textual Explanations of LLM Decisions via Attribution Guidance (2026.acl-long)
Copied to clipboard
| Challenge: | Prior work has focused on generating convincing rationales that appear to be subjectively faithful, but it remains unclear whether these explanations are epistemic faithful. |
| Approach: | They propose a method that enhances epistemic faithfulness by guiding explanation generation through attention-level interventions, informed by token-level heatmaps. |
| Outcome: | The proposed method significantly improves epistemic faithfulness across multiple models, benchmarks, and prompts. |
TensorLens: End-to-End Transformer Analysis via High-Order Attention Tensors (2026.acl-long)
Copied to clipboard
| Challenge: | Existing attention-aggregation methods focus on individual attention heads or layers, failing to account for the model’s global behavior. |
| Approach: | They propose a unified attention representation that captures the entire transformer as a single, input-dependent linear operator expressed through a high-order attention-interaction tensor. |
| Outcome: | The proposed model encapsulates the entire transformer as a single, input-dependent linear operator expressed through a high-order attention-interaction tensor. |
Focus Your Attention (with Adaptive IIR Filters) (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to deal with long-range data processing are implicit convolutions and regularized parameterization. |
| Approach: | They propose a new layer where dynamic (i.e., input-dependent) IIR filters are used to process the input sequence prior to applying conventional attention. |
| Outcome: | The proposed layer performs on-par with state-of-the-art networks with a fraction of their parameters and time complexity that is sub-quadratic with input size. |
The Hidden Attention of Mamba Models (2025.acl-long)
Copied to clipboard
| Challenge: | Recent studies have shown that Mamba models can be used for multiple domains, including NLP, long-range sequence processing, and computer vision. |
| Approach: | They add a third view and show that Mamba models can be viewed as attention-driven models. |
| Outcome: | The proposed model can be viewed as attention-driven and empirically compare it to the attention-based models of transformers. |