Papers by Tomer Ashuach
Masked by Consensus: Disentangling Privileged Knowledge in LLM Correctness (2026.acl-long)
Copied to clipboard
| Challenge: | Recent research suggests large language models encode meta-information about their own outputs. |
| Approach: | They investigate whether large language models possess similar privileged knowledge about answer correctness . they train correctness classifiers on question representations from a model’s hidden states and external models . |
| Outcome: | The proposed model outperforms peer-model models in factual knowledge tasks, but shows no advantage in math reasoning. |
CRISP: Persistent Concept Unlearning via Sparse Autoencoders (2026.acl-long)
Copied to clipboard
| Challenge: | Recent work has explored sparse autoencoders (SAEs) to perform precise interventions on monosemantic features, but most SAE-based methods operate at inference time, which does not create persistent changes in the model’s parameters. |
| Approach: | They propose a parameter-efficient method for persistent concept unlearning using SAEs that automatically identifies salient SAE features across multiple layers and suppresses their activations. |
| Outcome: | The proposed method outperforms previous methods on safety-critical unlearning tasks from the WMDP benchmark, successfully removing harmful knowledge while preserving general and in-domain capabilities. |
REVS: Unlearning Sensitive Information in Language Models via Rank Editing in the Vocabulary Space (2025.findings-acl)
Copied to clipboard
| Challenge: | Current approaches to address this issue involve costly dataset scrubbing or model filtering through unlearning and model editing. |
| Approach: | They propose a method for unlearning sensitive information from language models . they curate email and URL datasets and a social security number dataset . |
| Outcome: | The proposed method shows superior performance and robustness to extraction attacks on real-world datasets. |