Papers by Atticus Geiger

Updating CLIP to Prefer Descriptions Over Captions (2024.emnlp-main)

Copied to clipboard

Amir Zur, Elisa Kreiss, Karel D’Oosterlinck, Christopher Potts, Atticus Geiger

Challenge:	Current metrics for imagetext similarity tend to be insensitive to the text's purpose.
Approach:	They propose to use a model that assigns higher scores to descriptions than captions . they use parameter efficient fine-tuning and a loss objective to shed light on the distinction .
Outcome:	The proposed model correlates with the judgements of blind and low-vision people while preserving transfer capabilities and sheds light on the caption–description distinction.

pyvene: A Library for Understanding and Improving PyTorch Models via Interventions (2024.naacl-demo)

Copied to clipboard

Zhengxuan Wu, Atticus Geiger, Aryaman Arora, Jing Huang, Zheng Wang, Noah Goodman, Christopher Manning, Christopher Potts

Explainability & Interpretability in NLP

Challenge:	Existing libraries are often project-based, but pyvene provides a unified and extensible framework for performing interventions on neural models and sharing the intervened upon models with others.
Approach:	They propose an open-source Python library that supports customizable interventions on a range of different PyTorch modules.
Outcome:	The proposed framework provides a unified and extensible framework for performing interventions on neural models and sharing the intervened upon models with others.

RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representations (2024.acl-long)

Copied to clipboard

Jing Huang, Zhengxuan Wu, Christopher Potts, Mor Geva, Atticus Geiger

Semantic Text Processing

Challenge:	Existing methods to disentangle individual neurons from multiple high-level concepts are not yet benchmarked.
Approach:	They propose a method of Multi-task Distributed Alignment Search that allows to find distributed representations satisfying multiple causal criteria.
Outcome:	The proposed method achieves state-of-the-art on the target language model with Llama2-7B .

Dynabench: Rethinking Benchmarking in NLP (2021.naacl-main)

Copied to clipboard

Douwe Kiela, Max Bartolo, Yixin Nie, Divyansh Kaushik, Atticus Geiger, Zhengxuan Wu, Bertie Vidgen, Grusha Prasad, Amanpreet Singh, Pratik Ringshia, Zhiyi Ma, Tristan Thrush, Sebastian Riedel, Zeerak Waseem, Pontus Stenetorp, Robin Jia, Mohit Bansal, Christopher Potts, Adina Williams

Challenge:	Dynabench is an open-source platform for dynamic dataset creation and model benchmarking.
Approach:	They propose an open-source platform for dynamic dataset creation and model benchmarking.
Outcome:	The proposed platform can be used to create models that fail on simple challenges and falter in real-world scenarios.

Recursive Routing Networks: Learning to Compose Modules for Language Understanding (N19-1)

Copied to clipboard

Ignacio Cases, Clemens Rosenbaum, Matthew Riemer, Atticus Geiger, Tim Klinger, Alex Tamkin, Olivia Li, Sandhini Agarwal, Joshua D. Greene, Dan Jurafsky, Christopher Potts, Lauri Karttunen

Challenge:	Recursive Routing Networks are modular, adaptable models that learn effectively in diverse environments.
Approach:	They propose to apply Recursive Routing Networks (RRNs) to natural language understanding by integrating them into existing architectures and recurrent network hidden layers.
Outcome:	The proposed model optimizes the parameters of the functions and the meta-learner decision-making component for routing inputs through those functions.

DynaSent: A Dynamic Benchmark for Sentiment Analysis (2021.acl-long)

Copied to clipboard

Christopher Potts, Zhengxuan Wu, Atticus Geiger, Douwe Kiela

Sentiment Analysis

Challenge:	Sentiment analysis is an early success story for NLP, in both a technical and an industrial sense.
Approach:	They propose to combine naturally occurring sentences with sentences created using the open-source Dynabench Platform, which facilities human-and-model-in-the-loop dataset creation.
Outcome:	The proposed model is more coherent than comparable models and motivates training models from scratch over successive fine-tuning.

Causal Distillation for Language Models (2022.naacl-main)

Copied to clipboard

Zhengxuan Wu, Atticus Geiger, Joshua Rozner, Elisa Kreiss, Hanson Lu, Thomas Icard, Christopher Potts, Noah Goodman

Language Models

Challenge:	Distillation efforts have led to language models that are more compact and efficient without serious drops in performance.
Approach:	They propose to augment distillation with a third objective that encourages the student model to imitate the causal dynamics of the teacher through a distillation interchange intervention training objective (DIITO).
Outcome:	The proposed method lowers perplexity on the WikiText-103M corpus and improves on the GLUE benchmark, SQuAD, and CoNLL-2003.

Enhancing Automated Interpretability with Output-Centric Feature Descriptions (2025.acl-long)

Copied to clipboard

Yoav Gur-Arieh, Roy Mayan, Chen Agassy, Atticus Geiger, Mor Geva

Explainability & Interpretability in NLP

Challenge:	Existing pipelines for describing concepts in large language models (LLMs) rely on activating inputs to generate descriptions that fail to capture the causal effect of a feature on outputs.
Approach:	They propose to use inputs that activate features to generate natural language descriptions for concepts represented by large language models (LLMs) . identifying activating inputs is costly, and the mechanistic role of a feature in model behavior is determined by how inputs cause it to activate and how feature activation affects outputs.
Outcome:	The proposed methods capture the causal effect of a feature on outputs better than input-centric descriptions, but combine the two leads to the best performance on both input and output evaluations.

Posing Fair Generalization Tasks for Natural Language Inference (D19-1)

Copied to clipboard

Atticus Geiger, Ignacio Cases, Lauri Karttunen, Christopher Potts

Textual Inference

Challenge:	Existing evaluation methods for deep learning semantics rely on naturalistic corpora, but they often fail to support the kind of generalization we are asking for.
Approach:	They define and motivate a formal notion of fairness for evaluations of deep learning models for semantics . they then apply it to natural language inference by constructing challenging but provably fair artificial datasets based on the results .
Outcome:	The proposed evaluations show that standard neural models fail to generalize in the required ways and even these models do not solve the task perfectly.

ScoNe: Benchmarking Negation Reasoning in Language Models With Fine-Tuning and In-Context Learning (2023.acl-short)

Copied to clipboard

Jingyuan S. She, Christopher Potts, Samuel R. Bowman, Atticus Geiger

Semantic Text Processing

Challenge:	Negation is a ubiquitous but complex linguistic phenomenon that poses a significant challenge for NLP systems.
Approach:	They propose a benchmark that measures how well models handle natural language negation . they extend ScoNe-NLI to embed negation reasoning in short narratives .
Outcome:	The proposed model can reason about negation, but struggles to do so on NLI examples outside of its core pretraining regime.

Constructing Interpretable Features from Compositional Neuron Groups (2026.acl-long)

Copied to clipboard

Or David Shafran, Atticus Geiger, Mor Geva

Explainability & Interpretability in NLP

Challenge:	Existing methods for analyzing LLMs rely on dictionary learning with sparse autoencoders (SAEs) however, SAEs struggle in causal evaluations and lack intrinsic interpretability, as their learning is not explicitly tied to the computations of the model.
Approach:	They propose to decompose MLP activations with semi-nonnegative matrix factorization (SNMF) such that the learned features are mapped to their activating inputs, making them directly interpretable.
Outcome:	Experiments on Llama 3.1, Gemma 2 and GPT-2 show that SNMF derived features outperform SAEs and a strong supervised baseline on causal steering while aligning with human-interpretable concepts.