Papers by Mor Geva

LM vs LM: Detecting Factual Errors via Cross Examination (2023.emnlp-main)

Challenge:	Automated agents powered by large language models are becoming more ingrained into how people seek information . but evaluation benchmarks for LLMs rarely feature natural questions that are time-consuming . a new benchmark, MoNaCo, aims to address this gap by eliciting and manually answering time-wasting questions .
Approach:	They propose a benchmark of 1,315 natural and time-consuming questions that require dozens of intermediate steps to solve.
Outcome:	MoNaCo benchmarks achieve at least 61.2% F1 in real-world time-consuming questions hampered by low recall and hallucinations . Frontier LLMs evaluated on MoN achieving at least 61% F1, harmed by low memory and halluzinations.

Copied to clipboard

Roi Cohen, May Hamri, Mor Geva, Amir Globerson

Inferring Functionality of Attention Heads from their Parameters (2025.acl-long)

Challenge:	Modern language models (LMs) generate inconsistent, non-attributable or factually incorrect text, which hinders their usability.
Approach:	They propose a factuality evaluation framework for LMs that is based on cross-examination to detect inconsistencies between LM and examiner.
Outcome:	The proposed framework outperforms existing methods and baselines on factual claims on four benchmarks.

Copied to clipboard

Amit Elhelo, Mor Geva

Challenge:	Prior work on attention heads focused on analyzing their behavior during inference for specific circuits or tasks.
Approach:	They propose a framework that maps the functions of attention heads without any model training or inference.
Outcome:	The proposed framework infers the functionality of attention heads without any model training or inference.

Intrinsic Test of Unlearning Using Parametric Knowledge Traces (2025.emnlp-main)

Copied to clipboard

Yihuai Hong, Lei Yu, Haiqin Yang, Shauli Ravfogel, Mor Geva

Robustness in NLP

Challenge:	Existing methods for “unlearning” information captured in large language models rely on behavioral tests without monitoring residual knowledge in model parameters.
Approach:	They propose a general evaluation methodology that uses vocabulary projections to inspect concepts encoded in model parameters.
Outcome:	The proposed method detects changes in parametric traces of unlearned concepts and localizes them in two open-source LLMs.

LM-Debugger: An Interactive Tool for Inspection and Intervention in Transformer-Based Language Models (2022.emnlp-demos)

Copied to clipboard

Mor Geva, Avi Caciularu, Guy Dar, Paul Roit, Shoval Sadde, Micah Shlain, Bar Tamir, Yoav Goldberg

Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets (D19-1)

Challenge:	Transformer-based language models (LMs) are opaque and unexplained, causing problems for endusers and developers who wish to debug or fix their behaviour.
Approach:	They propose an interactive debugger tool for transformer-based LMs that provides a fine-grained interpretation of the model's internal prediction process and a powerful framework for intervening in LM behavior.
Outcome:	The proposed tool provides a fine-grained interpretation of the model's internal prediction construction process, and a powerful framework for intervening in LM behavior.

Copied to clipboard

Mor Geva, Yoav Goldberg, Jonathan Berant

Responsible & Trustworthy NLP

Challenge:	Having only a few workers generate the majority of dataset examples raises concerns about data diversity .
Approach:	They perform a series of experiments to investigate annotator biases in recent NLU datasets . they find that models are able to recognize the most productive annotators .
Outcome:	The results show that models can recognize the most productive annotators and do not generalize well to examples from annotator that did not contribute to the training set.

Transformer Feed-Forward Layers Are Key-Value Memories (2021.emnlp-main)

Copied to clipboard

Mor Geva, Roei Schuster, Jonathan Berant, Omer Levy

From Insights to Actions: The Impact of Interpretability and Analysis Research on NLP (2024.emnlp-main)

Challenge:	Using feed-forward layers, we show that the learned patterns are human-interpretable, and that lower layers tend to capture shallow patterns, while upper layers learn more semantic ones.
Approach:	They propose that feed-forward layers in transformer-based language models operate as key-value memories where each key correlates with textual patterns in the training examples and each value induces a distribution over the output vocabulary.
Outcome:	The proposed model is based on key-value memories with a key-level correlation with the training examples and a distribution over the output vocabulary.

Copied to clipboard

Marius Mosbach, Vagrant Gautam, Tomás Vergara Browne, Dietrich Klakow, Mor Geva

Responsible & Trustworthy NLP

Challenge:	Interpretability and analysis (IA) research is a growing subfield within NLP . a criticism of this work is that it lacks actionable insights and therefore has little impact on NLP.
Approach:	They propose to quantify the impact of interpretation and analysis research on NLP . they use citation graphs and a survey to find out what is missing in IA research .
Outcome:	The proposed study shows that IA research is well-cited outside of IA and central in the NLP citation graph.

Hopping Too Late: Exploring the Limitations of Large Language Models on Multi-Hop Queries (2024.emnlp-main)

Copied to clipboard

Eden Biran, Daniela Gottesman, Sohee Yang, Mor Geva, Amir Globerson

RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representations (2024.acl-long)

Challenge:	Large language models (LLMs) can solve complex multi-step problems, but little is known about how these computations are implemented internally.
Approach:	They propose a "back-patching" analysis method to solve multi-hop queries . they propose resolving the bridge entity into the bridge and the second hop into the target entity into latent steps.
Outcome:	The proposed method solves multi-hop queries that require two information extraction steps . it shows that the later layers lack the necessary knowledge to correctly generate the answer .

Copied to clipboard

Jing Huang, Zhengxuan Wu, Christopher Potts, Mor Geva, Atticus Geiger

Do Large Language Models Perform Latent Multi-Hop Reasoning without Exploiting Shortcuts? (2025.findings-acl)

Challenge:	Existing methods to disentangle individual neurons from multiple high-level concepts are not yet benchmarked.
Approach:	They propose a method of Multi-task Distributed Alignment Search that allows to find distributed representations satisfying multiple causal criteria.
Outcome:	The proposed method achieves state-of-the-art on the target language model with Llama2-7B .

Copied to clipboard

Sohee Yang, Nora Kassner, Elena Gribovskaya, Sebastian Riedel, Mor Geva

Crawling The Internal Knowledge-Base of Language Models (2023.findings-eacl)

Challenge:	Latent multi-hop reasoning is a problem in Large Language Models that can develop shortcuts by encountering the head entity and answer entity in training sequences.
Approach:	They propose desiderata for shortcut-free evaluation of latent multi-hop reasoning ability . they exclude test queries where head and answer entities might have co-appeared .
Outcome:	The proposed model can latently recall and compose single-hop facts without shortcuts, but only for certain types of queries.

Copied to clipboard

Roi Cohen, Mor Geva, Jonathan Berant, Amir Globerson

What’s in Your Head? Emergent Behaviour in Multi-Task Transformer Models (2021.emnlp-main)

Challenge:	Existing methods for representing factual knowledge in a language model are insufficient.
Approach:	They propose a procedure for “crawling” the internal knowledge-base of a language model by expanding a knowledge-graph around it.
Outcome:	The proposed method yields high precision graphs (82-92%) while emitting a reasonable number of facts per entity.

Copied to clipboard

Mor Geva, Uri Katz, Aviv Ben-Arie, Jonathan Berant

Don’t Blame the Annotator: Bias Already Starts in the Annotation Instructions (2023.eacl-main)

Challenge:	Existing paradigms for multi-task training involve a shared pre-trained language model and a small, thin network (head) given an input, a target head is the head that is selected for outputting the final prediction.
Approach:	They examine the behaviour of non-target heads when given input that belongs to a different task than the one they were trained for.
Outcome:	The non-target heads exhibit emergent behaviour, which may explain the target task, or generalize beyond their original task.

Copied to clipboard

Mihir Parmar, Swaroop Mishra, Mor Geva, Chitta Baral

Responsible & Trustworthy NLP

Challenge:	Recent studies have shown that data collected through crowdsourcing often exhibit various biases that lead to overestimation of model performance.
Approach:	They propose to model instruction bias in 14 recent NLU benchmarks by analyzing crowdsourcing instructions and analyzing their results.
Outcome:	The proposed model can be over-represented in datasets with a large number of examples, and the results are consistent with previous studies.

Break It Down: A Question Understanding Benchmark (2020.tacl-1)

Copied to clipboard

Tomer Wolfson, Mor Geva, Ankit Gupta, Matt Gardner, Yoav Goldberg, Daniel Deutch, Jonathan Berant

Question Answering

Challenge:	Understanding natural language questions entails the ability to break down a question into the requisite steps for computing its answer.
Approach:	They introduce a Question Decomposition Meaning Representation (QDMR) for questions . they demonstrate that QDMRs can be annotated at scale using a hotpotQA dataset .
Outcome:	The proposed model outperforms several natural baselines in the open-domain question answering hotpotQA dataset and can be deterministically converted to a pseudo-SQL formal language.

Injecting Numerical Reasoning Skills into Language Models (2020.acl-main)

Copied to clipboard

Mor Geva, Ankit Gupta, Jonathan Berant

Complex Reasoning in Natural Language (2023.acl-tutorials)

Challenge:	Existing models for numerical reasoning are limited by their flexibility and require specialized architectures to capture high-level skills.
Approach:	They propose to inject numerical reasoning skills into pre-trained LMs by generating large amounts of data and training in a multi-task setup.
Outcome:	The proposed model performs better on DROP than other models of comparable size while maintaining high performance on standard RC tasks.

Copied to clipboard

Wenting Zhao, Mor Geva, Bill Yuchen Lin, Michihiro Yasunaga, Aman Madaan, Tao Yu

Learning to Search in Long Documents Using Document Structure (C18-1)

Challenge:	Recent research shows that pretrained language models are often brittle for complex reasoning tasks.
Approach:	They propose to use pre-trained language models to teach machines to reason over texts . they will review recent promising approaches to tackling complex reasoning tasks .
Outcome:	This tutorial reviews promising approaches to complex reasoning tasks . it reviews the methods that can be used to augment models with robustness .

Copied to clipboard

Mor Geva, Jonathan Berant

Question Answering

Challenge:	Reading comprehension models are dominated by recurrent neural networks (RNNs) as documents become longer and questions become complex, sequential reading becomes a significant bottleneck.
Approach:	They propose a reading comprehension framework that uses document trees to model an agent that interleaves quick navigation with more expensive answer extraction.
Outcome:	The proposed model improves question answering performance compared to existing models and has a strong information-retrieval baseline.

Performance Gap in Entity Knowledge Extraction Across Modalities in Vision Language Models (2025.acl-long)

Copied to clipboard

Ido Cohen, Daniela Gottesman, Mor Geva, Raja Giryes

Multimodality

Challenge:	Vision-language models excel at extracting and reasoning about information from images, yet their capacity to leverage internal knowledge about specific entities remains underexplored.
Approach:	They propose a dataset which allows separating entity recognition and question answering . they hypothesize that this decline arises from limitations in how information flows from image tokens to query tokens.
Outcome:	The proposed model performance drops when the entity is presented visually rather than textually.

Eliciting Textual Descriptions from Representations of Continuous Prompts (2025.findings-acl)

Copied to clipboard

Daniela Gottesman, Mor Geva, Dana Ramati

Can Large Language Models Faithfully Express Their Intrinsic Uncertainty in Words? (2024.emnlp-main)

Challenge:	Prior attempts to interpret continuous prompts relied on projecting individual tokens onto the vocabulary space, but this approach yielded arbitrary or contradictory text.
Approach:	They propose to project individual prompt tokens onto the model's vocabulary space and interpret each prompt token individually.
Outcome:	The proposed method yields accurate task descriptions and is more faithful as task performance increases.

Copied to clipboard

Gal Yona, Roee Aharoni, Mor Geva

Narrowing the Knowledge Evaluation Gap: Open-Domain Question Answering with Multi-Granularity Answers (2024.acl-long)

Challenge:	Despite their unprecedented capabilities, large language models (LLMs) often output erroneous information, which may lead users to overly rely on their false output.
Approach:	They formalize faithful response uncertainty based on the gap between the model’s intrinsic confidence in the assertions it makes and the decisiveness by which they are conveyed.
Outcome:	The proposed model is poor at faithfully conveying uncertainty on knowledge-intensive questions.

Copied to clipboard

Gal Yona, Roee Aharoni, Mor Geva

Natural Language Interfaces

Challenge:	Standard question answering evaluation protocols do not take factual errors into account and compare a predicted answer against answers of a single granularity level.
Approach:	They propose a new evaluation setting where a predicted answer is evaluated in terms of accuracy and informativeness against a set of multi-granularity answers.
Outcome:	The proposed evaluation setting compares a predicted answer against a set of multi-granularity answers.

Dissecting Recall of Factual Associations in Auto-Regressive Language Models (2023.emnlp-main)

Copied to clipboard

Mor Geva, Jasmijn Bastings, Katja Filippova, Amir Globerson

Detecting (Un)answerability in Large Language Models with Linear Directions (2026.eacl-long)

Challenge:	Existing studies have focused on identifying where factual knowledge is encoded in the network, but little is known about how it is extracted from the model parameters during inference.
Approach:	They examine how factual associations are stored and retrieved internally in LMs . they use attention edges to identify critical points where information propagates to the prediction .
Outcome:	The proposed model aggregates information about subject and relation to predict the correct attribute . the model “queries” the enriched subject to extract the attribute based on the proposed model .

Copied to clipboard

Maor Juliet Lavi, Tova Milo, Mor Geva

Information Retrieval

Challenge:	Large language models (LLMs) often respond confidently to questions even when they lack the necessary information, leading to inaccurate responses or hallucinations.
Approach:	They propose an approach for identifying a direction in the model’s activation space that captures unanswerability and uses it for classification.
Outcome:	The proposed method detects unanswerable questions and generalizes better across datasets than existing prompt-based and classifier-based approaches.

A Comprehensive Evaluation of Tool-Assisted Generation Strategies (2023.findings-emnlp)

Copied to clipboard

Alon Jacovi, Avi Caciularu, Jonathan Herzig, Roee Aharoni, Bernd Bohnet, Mor Geva

Understanding Transformer Memorization Recall Through Idioms (2023.eacl-main)

Challenge:	Various few-shot tool-usage strategies have been proposed to overcome LMs' shortcomings.
Approach:	They propose to augment language models with tools to overcome their shortcomings . they find strong no-tool baselines are competitive to tool-assisted strategies .
Outcome:	The proposed strategies outperform those that refine incorrect outputs with tools in knowledge-retrieval tasks, the study finds . the findings suggest few-shot tool integration is still an open challenge .

Copied to clipboard

Adi Haviv, Ido Cohen, Jacob Gidron, Roei Schuster, Yoav Goldberg, Mor Geva

Inferring Implicit Relations in Complex Questions with Language Models (2022.findings-emnlp)

Challenge:	Existing methods for analyzing memorization use definitions that are based on model performance, which changes between models and often also between training runs.
Approach:	They propose idioms as inputs that typically trigger memory recall and propose a set of English idiomas to test their methodological framework for probing and characterizing recall of memorized sequences in transformer LMs.
Outcome:	The proposed framework compares model behavior on memorized vs. non-memorized inputs across different model sizes and architectures.

Copied to clipboard

Uri Katz, Mor Geva, Jonathan Berant

Question Answering

Challenge:	A prominent challenge for language understanding systems is the ability to answer implicit reasoning questions where the evidence for answering the question is not mentioned explicitly.
Approach:	They propose to decouple inference of reasoning steps from execution by evaluating models of implicit relation inference.
Outcome:	The proposed model fails on the implicit reasoning QA task, but infers implicit relations . the proposed model is compared with other models that fail on the same task .

How Well Can Reasoning Models Identify and Recover from Unhelpful Thoughts? (2025.findings-emnlp)

Copied to clipboard

Sohee Yang, Sang-Woo Lee, Nora Kassner, Daniela Gottesman, Sebastian Riedel, Mor Geva

A Chain-of-Thought Is as Strong as Its Weakest Link: A Benchmark for Verifiers of Reasoning Chains (2024.acl-long)

Challenge:	Recent reasoning models show the ability to reflect, backtrack, and self-validate their reasoning .
Approach:	They investigate how well models identify and recover from unhelpful thoughts . they show that models are effective at identifying most unhelpfully thoughts - but struggle to recover from the same thoughts when they are injected into their thinking process .
Outcome:	The proposed model can identify unhelpful thoughts but struggle to recover from them . larger models struggle more than smaller models to recover short irrelevant thoughts .

Copied to clipboard

Alon Jacovi, Yonatan Bitton, Bernd Bohnet, Jonathan Herzig, Or Honovich, Michael Tseng, Michael Collins, Roee Aharoni, Mor Geva

DiscoFuse: A Large-Scale Dataset for Discourse-Based Sentence Fusion (N19-1)

Challenge:	Recent literature discusses automatic methods to evaluate reasoning to improve their correctness, but no fine-grained step-level datasets are available to enable thorough evaluation of such verification methods.
Approach:	They propose to benchmark automatic verifiers of complex Chain-of-Thought reasoning in open-domain question-answering settings using a dataset that includes comprehensive labels for relevance, attribution to evidence passages, and logical correctness of each reasoning step.
Outcome:	The proposed dataset shows that verifiers struggle at verifying reasoning chains, particularly verifying logical correctness and detecting contradictions.

Copied to clipboard

Mor Geva, Eric Malmi, Idan Szpektor, Jonathan Berant

Challenge:	Existing datasets for sentence fusion are small and insufficient for training modern neural models.
Approach:	They propose a method for automatically-generating fusion examples from raw text . they apply their method to Wikipedia and Sports articles to generate fusion models .
Outcome:	The proposed method improves performance on WebSplit when viewed as a sentence fusion task.

SCROLLS: Standardized CompaRison Over Long Language Sequences (2022.emnlp-main)

Copied to clipboard

Uri Shaham, Elad Segal, Maor Ivgi, Avia Efrat, Ori Yoran, Adi Haviv, Ankit Gupta, Wenhan Xiong, Mor Geva, Jonathan Berant, Omer Levy

Challenge:	Standard NLP benchmarks focus on short texts, but long texts are produced in the context of longer discourses.
Approach:	They propose a new benchmark that places models in context of long texts that require reasoning over long texts.
Outcome:	The proposed task sets are based on a set of long-text datasets and host a live leaderboard to facilitate research on model architecture and pretraining methods.

Backward Lens: Projecting Language Model Gradients into the Vocabulary Space (2024.emnlp-main)

Copied to clipboard

Shahar Katz, Yonatan Belinkov, Mor Geva, Lior Wolf

Precise In-Parameter Concept Erasure in Large Language Models (2025.emnlp-main)

Challenge:	Recent interpretability methods project weights and hidden states obtained from the forward pass to the models’ vocabularies, helping to uncover how information flows within LMs.
Approach:	They propose to cast a gradient matrix as a low-rank linear combination of forward and backward passes’ inputs and then to project these gradients into vocabulary items.
Outcome:	The proposed method can be cast as a low-rank linear combination of forward and backward passes’ inputs and project these gradients into vocabulary items.

Copied to clipboard

Yoav Gur-Arieh, Clara Haya Suslik, Yihuai Hong, Fazl Barez, Mor Geva

CRoW: Benchmarking Commonsense Reasoning in Real-World Tasks (2023.emnlp-main)

Challenge:	Existing methods for erasing concepts from model parameters are either too coarse, shallow or ineffective.
Approach:	They propose a framework for precisely erasing entire concepts from model parameters by directly editing directions that encode them in parameter space.
Outcome:	The proposed framework reduces errors on the target concept to as low as 7.7% while improving specificity and robustness.

Copied to clipboard

Mete Ismayilzada, Debjit Paul, Syrielle Montariol, Mor Geva, Antoine Bosselut

Jump to Conclusions: Short-Cutting Transformers with Linear Transformations (2024.lrec-main)

Challenge:	Recent efforts in natural language processing (NLP) commonsense reasoning research have produced a number of new datasets and benchmarks.
Approach:	They propose a manually-curated, multi-task benchmark that evaluates models' ability to apply commonsense reasoning in the context of six real-world NLP tasks.
Outcome:	The proposed benchmark evaluates the ability of models to apply commonsense reasoning in the context of six real-world NLP tasks.

Copied to clipboard

Alexander Yom Din, Taelin Karidi, Leshem Choshen, Mor Geva

Language Models Encode Numbers Using Digit Representations in Base 10 (2025.naacl-short)

Challenge:	Transformer-based language models create hidden representations of inputs at every layer, but only use final-layer representations for prediction.
Approach:	They propose a method for casting hidden representations as final representations, bypassing transformer computation in-between.
Outcome:	The proposed method produces more accurate predictions from hidden layers across various model scales, architectures, and data distributions.

Copied to clipboard

Amit Arnold Levy, Mor Geva

Enhancing Automated Interpretability with Output-Centric Feature Descriptions (2025.acl-long)

Challenge:	Large language models (LLMs) often make errors when handling simple numerical tasks . a natural hypothesis is that these errors stem from how LLMs represent numbers .
Approach:	They propose to examine how LLMs represent numbers with circular representations per digit . they propose to use digit-wise representations to shed light on errors on numerical tasks .
Outcome:	The proposed model is internally represented with individual circular representations per-digit in base 10 . the proposed model could be used to analyze numerical mechanisms in large language models .

Copied to clipboard

Yoav Gur-Arieh, Roy Mayan, Chen Agassy, Atticus Geiger, Mor Geva

Explainability & Interpretability in NLP

Challenge:	Existing pipelines for describing concepts in large language models (LLMs) rely on activating inputs to generate descriptions that fail to capture the causal effect of a feature on outputs.
Approach:	They propose to use inputs that activate features to generate natural language descriptions for concepts represented by large language models (LLMs) . identifying activating inputs is costly, and the mechanistic role of a feature in model behavior is determined by how inputs cause it to activate and how feature activation affects outputs.
Outcome:	The proposed methods capture the causal effect of a feature on outputs better than input-centric descriptions, but combine the two leads to the best performance on both input and output evaluations.

Do Large Language Models Latently Perform Multi-Hop Reasoning? (2024.acl-long)

Copied to clipboard

Sohee Yang, Elena Gribovskaya, Nora Kassner, Mor Geva, Sebastian Riedel

Estimating Knowledge in Large Language Models Without Generating a Single Token (2024.emnlp-main)

Challenge:	Large Language Models (LLMs) latently perform multi-hop reasoning with complex prompts such as "The mother of the singer of ‘Superstition’ is".
Approach:	They examine whether Large Language Models (LLMs) latently perform multi-hop reasoning with complex prompts such as "The mother of the singer of ‘Superstition’ is".
Outcome:	The proposed model can latently perform multi-hop reasoning with complex prompts such as "The mother of the singer of ‘Superstition’ is".

Copied to clipboard

Daniela Gottesman, Mor Geva

Break, Perturb, Build: Automatic Perturbation of Reasoning Paths Through Question Decomposition (2022.tacl-1)

Challenge:	Existing methods to evaluate knowledge in large language models require querying and evaluating the model's generated responses.
Approach:	They ask whether it is possible to estimate how knowledgeable a model is about a subject entity only from its internal computation.
Outcome:	The proposed model performs well with QA accuracy and FActScore . it can be leveraged to guide decisions on how to apply further training or augment queries with retrieval.

Copied to clipboard

Mor Geva, Tomer Wolfson, Jonathan Berant

The Hidden Space of Transformer Language Adapters (2024.acl-long)

Challenge:	Recent efforts to create challenge benchmarks that test the abilities of natural language understanding models have largely depended on human annotations.
Approach:	They propose a framework for automatic reasoning-oriented perturbation of question-answer pairs that decomposes a question into reasoning steps required to answer it and generates new question-anchor pairs.
Outcome:	The proposed framework generates evaluation sets for reading comprehension benchmarks and generates examples without human intervention.

Copied to clipboard

Jesujoba Alabi, Marius Mosbach, Matan Eyal, Dietrich Klakow, Mor Geva

Constructing Interpretable Features from Compositional Neuron Groups (2026.acl-long)

Challenge:	Adapters are small modules trained on top of a frozen language model to adapt predictions to new target languages.
Approach:	They propose to train transformer language adapters on top of a frozen model to adapt predictions to new target languages.
Outcome:	The transformer language adapters are trained on top of a frozen model to adapt predictions to new target languages.

Copied to clipboard

Or David Shafran, Atticus Geiger, Mor Geva

Explainability & Interpretability in NLP

Challenge:	Existing methods for analyzing LLMs rely on dictionary learning with sparse autoencoders (SAEs) however, SAEs struggle in causal evaluations and lack intrinsic interpretability, as their learning is not explicitly tied to the computations of the model.
Approach:	They propose to decompose MLP activations with semi-nonnegative matrix factorization (SNMF) such that the learned features are mapped to their activating inputs, making them directly interpretable.
Outcome:	Experiments on Llama 3.1, Gemma 2 and GPT-2 show that SNMF derived features outperform SAEs and a strong supervised baseline on causal steering while aligning with human-interpretable concepts.

In-Context Learning Creates Task Vectors (2023.findings-emnlp)

Copied to clipboard

Roee Hendel, Mor Geva, Amir Globerson

Analyzing Transformers in Embedding Space (2023.acl-long)

Challenge:	In-context learning (ICL) is a powerful new learning paradigm for Large Language Models (LLMs).
Approach:	They propose to use a model with a prompt and a query to learn a mapping based on two examples to produce the output.
Outcome:	The proposed model can learn functions from a simple structure based on a training set and a single task vector calculated from the training set.

Copied to clipboard

Guy Dar, Mor Geva, Ankit Gupta, Jonathan Berant

Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space (2022.emnlp-main)

Challenge:	Understanding Transformer-based models has attracted significant attention . a zero-pass approach is feasible for some parameters, and for two-layer attention networks .
Approach:	They propose a theoretical framework where parameters of a trained Transformer are interpreted by projecting them into the embedding space.
Outcome:	The proposed framework shows that pre-trained and fine-tuned models can be interpreted in embedding space.

Copied to clipboard

Mor Geva, Avi Caciularu, Kevin Wang, Yoav Goldberg