Papers by Ramon Huerta
UNDIAL: Self-Distillation with Adjusted Logits for Robust Unlearning in Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for unlearning large language models fine-tune by maximizing loss, but they are unstable . this creates instability, especially on larger datasets, leading to over-unlearning . |
| Approach: | They propose a novel unlearning method that leverages self-distillation to adjust logits . this method ensures smooth convergence and avoids catastrophic forgetting . |
| Outcome: | The proposed method achieves smooth convergence and avoids catastrophic forgetting even on large datasets and sequential unlearning requests. |