Papers by Fazl Barez
Towards Interpretable Sequence Continuation: Analyzing Shared Circuits in Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent work aims to reverse engineer transformer models into human-readable representations . transformers exhibit strong capabilities on linguistic tasks, but their complex architectures make them difficult to interpret. |
| Approach: | They extend transformer models into human-readable representations that implement algorithmic functions by analyzing sequence continuation tasks. |
| Outcome: | The proposed model can be reverse-engineered into human-readable representations that implement algorithmic functions. |
The Larger they are, the Harder they Fail: Language Models do not Recognize Identifier Swaps in Python (2023.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are used for programming tasks but lack a deep understanding of the content they manipulate. |
| Approach: | They show that LLMs fail to correctly generate correct Python code when default function names are swapped . they also show that they become more confident in their incorrect predictions as the model size increases . |
| Outcome: | The proposed models fail to generate correct Python code when default function names are swapped, and become more confident in their incorrect predictions as the model size increases. |
Interpreting Context Look-ups in Transformers: Investigating Attention-MLP Interactions (2024.emnlp-main)
Copied to clipboard
| Challenge: | Using a method to identify next-token neurons, we find that some attention heads recognize contexts relevant to predicting a token and activate a downstream token-predicting neuron accordingly. |
| Approach: | They propose a method to identify next-token neurons and determine the upstream attention heads responsible for their activity in LLMs. |
| Outcome: | The proposed method identifies next-token neurons, finds prompts that highly activate them, and determines the upstream attention heads responsible. |
Same Question, Different Words: A Latent Adversarial Framework for Prompt Robustness (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing solutions to the problem of semantically-preserving variations of prompts are expensive and require trial-and-error prompt engineering. |
| Approach: | They propose a dual-loop adversarial framework that optimizes a trainable perturbation as "latent continuous paraphrase" they demonstrate a 0.5%-4% improvement on worst-case win-rate on the RobustAlpaca benchmark . |
| Outcome: | Extensive experiments show that the proposed framework improves on the RobustAlpaca benchmark with a 0.5%-4% improvement on the worst-case win-rate. |
Large Language Models Relearn Removed Concepts (2024.findings-acl)
Copied to clipboard
| Challenge: | a recent study has shown that models can re-acquire pruned concepts after editing . however, it remains unclear whether models can recover such concepts after retraining . |
| Approach: | They evaluate concept relearning in large language models by tracking concept saliency and similarity in pruned neurons during retraining for named entity recognition tasks. |
| Outcome: | The results show that models can re-acquire pruned concepts after pruning . they also show that they can blend old and new concepts in individual neurons . |
Beyond Linear Steering: Unified Multi-Attribute Control for Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Empirical results show that K-Steering outperforms strong baselines in accurately steering multiple behaviors. |
| Approach: | They propose a method that trains a single classifier on hidden activations and computes intervention directions via gradients at inference time. |
| Outcome: | The proposed method outperforms strong baselines in steering multiple behaviors. |
Precise In-Parameter Concept Erasure in Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for erasing concepts from model parameters are either too coarse, shallow or ineffective. |
| Approach: | They propose a framework for precisely erasing entire concepts from model parameters by directly editing directions that encode them in parameter space. |
| Outcome: | The proposed framework reduces errors on the target concept to as low as 7.7% while improving specificity and robustness. |
Trust Me, I’m Wrong: LLMs Hallucinate with Certainty Despite Knowing the Answer (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work on large language model (LLM) hallucinations associated with model uncertainty or inaccurate knowledge. |
| Approach: | They define and investigate a type of hallucination where a model can answer a question correctly but a perturbation causes it to produce a hallucinous response with high certainty. |
| Outcome: | The proposed mitigations outperform existing methods on CHOKE hallucinations . the findings highlight the need to understand their origins and improve mitigation strategies . |
Make Mechanistic Interpretability Auditable: A Call to Develop Guidelines via Continuous Collaborative Reviewing (2026.acl-long)
Copied to clipboard
Michael Lan, Narmeen Fatimah Oozeer, Chaithanya Bandi, Philip Quirke, Austin Meek, Fazl Barez, Amir Abdullah
| Challenge: | a recent paper found conflicting conclusions for the same behavior in a neural network . authors propose auditing MI itself is essential for its application in AI safety, industry, and governance . |
| Approach: | They propose to develop a system that can audit experiments to ensure validity . authors propose to generalize good practices found on platform into expert-verified guidelines . |
| Outcome: | a new review system could be developed that can be standardized and audited . authors argue that auditing MI is essential for its application in AI safety, industry, and governance . |
Detecting Edit Failures In Large Language Models: An Improved Specificity Benchmark (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent model editing techniques can introduce large unwanted side effects, a new study shows . existing specificity benchmarks do not detect these unwanted side-effects . a recent study shows that model edits can cause significant performance drop . |
| Approach: | They extend existing CounterFact benchmark to include a dynamic component and propose a new benchmark to evaluate model editing techniques. |
| Outcome: | The proposed benchmark improves existing benchmarks for specificity and avoids unwanted side effects. |