Papers by Shauli Ravfogel
Contrastive Explanations for Model Interpretability (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for producing model explanations seek all causal factors at once, making them difficult to comprehend. |
| Approach: | They propose a method to produce contrastive explanations in the latent space . they use attribution and token/span attribution to produce models that consider only contrastive reasoning . |
| Outcome: | The proposed method allows model behavior to consider only contrastive reasoning . it also uncovers which aspects of the input are useful for and against particular decisions . |
Adversarial Concept Erasure in Kernel Space (2022.emnlp-main)
Copied to clipboard
| Challenge: | Large neural networks in NLP produce real-valued representations that encode the bit of human language that they were trained on. |
| Approach: | They propose a kernelization of the recently-proposed linear concept-removal objective and propose to remove linear subspaces from the representation space. |
| Outcome: | The proposed kernelization protects against the ability of nonlinear adversaries to recover the concept. |
Intrinsic Test of Unlearning Using Parametric Knowledge Traces (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for “unlearning” information captured in large language models rely on behavioral tests without monitoring residual knowledge in model parameters. |
| Approach: | They propose a general evaluation methodology that uses vocabulary projections to inspect concepts encoded in model parameters. |
| Outcome: | The proposed method detects changes in parametric traces of unlearned concepts and localizes them in two open-source LLMs. |
Language Concept Erasure for Language-invariant Dense Retrieval (2024.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual models aim for language-invariant representations but still encode language identity. |
| Approach: | They propose a multi-task learning framework that induces language invariance in multilingual retrieval by reducing language-specific signals in the embedding space. |
| Outcome: | The proposed learning framework improves language-invariant dense retrieval over baselines on English retrieval data and general multilingual corpora. |
Guiding LLM to Fool Itself: Automatically Manipulating Machine Reading Comprehension Shortcut Triggers (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent applications of LLMs in Machine Reading Comprehension systems have shown impressive results, but the use of shortcuts has emerged as a potential threat to their reliability. |
| Approach: | They propose a framework that guides an editor to add potential shortcuts-triggers to samples. |
| Outcome: | The proposed framework can edit trigger shortcuts in samples that fool LLMs . it also shows that GPT4 can be deceived by its own edits (15% drop in F1). |
The Curious Case of Hallucinatory (Un)answerability: Finding Truths in the Hidden States of Over-Confident Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have been shown to possess impressive capabilities, but they are not problem-free. |
| Approach: | They explore the behavior of large language models when presented with (un)answerable queries. |
| Outcome: | The proposed models encode the answerability of an input query, the authors show . they also show that the first decoded token is a strong indicator . |
Studying the Inductive Biases of RNNs with Synthetic Variations of Natural Languages (N19-1)
Copied to clipboard
| Challenge: | Recent studies have identified both strengths and limitations of recurrent neural networks (RNNs) in applied natural language processing tasks. |
| Approach: | They propose a paradigm that addresses typological differences between languages . they create synthetic versions of English and train them to predict agreement features . |
| Outcome: | The proposed model improves on predicting agreement with subject and object, suggesting that RNNs have a recency bias. |
Neural Extractive Search (2021.acl-demo)
Copied to clipboard
| Challenge: | a domain expert often needs to extract structured information from large corpora. |
| Approach: | They propose a search paradigm called "extractive search" that extends search queries with capture-slots to allow for rapid extraction. |
| Outcome: | The proposed search paradigm can be extended with neural similarity techniques. |
Measuring and Improving Consistency in Pretrained Language Models (2021.tacl-1)
Copied to clipboard
Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hinrich Schütze, Yoav Goldberg
| Challenge: | In this paper, we examine whether pretrained language models are consistent with factual knowledge. |
| Approach: | They propose a method to improve consistency of pretrained language models . consistency is a desirable property of a good language understanding model, they argue . |
| Outcome: | The proposed model improves consistency and shows that it is effective. |
BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models (2022.acl-short)
Copied to clipboard
| Challenge: | Large pre-trained models are expensive to train and deploy . large data sets make finetuning expensive to deploy - a new paradigm . |
| Approach: | They propose a sparse-finetuning method where only bias-terms are modified . they show that applying BitFit on pre-trained BERT models is competitive . |
| Outcome: | The proposed method is competitive with sparse-finetuning methods on small-to-medium training data. |
Erratum: Measuring and Improving Consistency in Pretrained Language Models (2021.tacl-1)
Copied to clipboard
Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hinrich Schütze, Yoav Goldberg
| Challenge: | During production of this paper, an error was introduced to the formula on the bottom of the right column of page 1020. |
| Approach: | the formula was changed in the last two terms of the paper . |
| Outcome: | the correct formula is now available on the web. |
Ab Antiquo: Neural Proto-language Reconstruction (2021.naacl-main)
Copied to clipboard
| Challenge: | Historical linguists have identified regularities in the process of historic sound change. |
| Approach: | They propose a method to reconstruct proto-words based on cognates in daughter languages . they use a dataset of 8,000 comparative entries to analyze phonological changes . |
| Outcome: | The proposed method outperforms conventional methods in a proto-word reconstruction task. |
Log-linear Guardedness and its Implications (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for erasing human-interpretable concepts from neural representations that assume linearity are not fully understood. |
| Approach: | They define linear guardedness as the inability of an adversary to predict the concept directly from the representation . they show that a log-linear model can be constructed that indirectly recovers the concept . |
| Outcome: | The proposed model can be constructed that indirectly recovers the erased concept in some cases. |
Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals (2021.tacl-1)
Copied to clipboard
| Challenge: | Amnesic probing is a method that focuses on how information is being used, rather than on what information is encoded. |
| Approach: | They propose a method that focuses on how the information is being used rather than on what information is encoded. |
| Outcome: | The proposed method is based on a BERT dataset to ask questions that were not possible before . it shows that probing performance is not correlated to task importance . |
The Medium Is Not the Message: Deconfounding Document Embeddings via Linear Concept Erasure (2025.emnlp-main)
Copied to clipboard
| Challenge: | Embedding-based similarity metrics can be influenced by content dimensions and spurious attributes like the text’s source or language. |
| Approach: | They propose a debiasing algorithm that removes observed confounders from encoder representations and removes them from the encoder. |
| Outcome: | The proposed method improves on out-of-distribution benchmarks and on benchmarks, but performance is not affected. |
Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection (2020.acl-main)
Copied to clipboard
| Challenge: | Word embeddings, pre-trained language models, and deep learning methods are becoming effective for text classification. |
| Approach: | They propose a method for removing information from neural representations using null-space projection. |
| Outcome: | The proposed method mitigates bias in word embeddings and increases fairness in multi-class classification. |
Conformal Nucleus Sampling (2023.findings-acl)
Copied to clipboard
| Challenge: | Modern language generation methods employ one of a handful of standard decoding strategies . greedy search and other methods generate dull text or degenerate text . |
| Approach: | They employ a calibration procedure to calibrate the parameter p as a function of the entropy of the next word distribution. |
| Outcome: | The proposed method overconfidently selects the word with the highest probability . the greedy method and its beam search variations tend to return dull text or degenerate text . |
A Practical Method for Generating String Counterfactuals (2025.findings-naacl)
Copied to clipboard
| Challenge: | Interventions targeting the representation space of language models (LMs) have emerged as an effective means to influence model behavior. |
| Approach: | They propose a method to convert representation counterfactuals into string counterf actuals and analyze the linguistic alterations corresponding to the intervention. |
| Outcome: | The proposed method analyzes linguistic alterations corresponding to a given representation space intervention and interprets features utilized to encode a specific concept. |
Few-shot Fine-tuning vs. In-context Learning: A Fair Comparison and Evaluation (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies show that in-context learning and few-shot fine-tuning can generalize well out-of-domain. |
| Approach: | They compare few-shot fine-tuning and in-context learning for task adaptation . they find that both approaches generalize similarly, but exhibit large variation . |
| Outcome: | The proposed methods outperform in-context learning and few-shot fine-tuning with OPT models of different sizes. |