Papers by Stefan Vasilev
Unilogit: Robust Machine Unlearning for LLMs Using Uniform-Target Self-Distillation (2025.findings-acl)
Copied to clipboard
| Challenge: | Extensive experiments on public benchmarks and an in-house e-commerce dataset demonstrate Unilogit’s superior performance in balancing forget and retain objectives, outperforming state-of-the-art methods such as NPO and UnDIAL. |
| Approach: | They propose a self-distillation method that dynamically adjusts target logits to achieve a uniform probability for the target token. |
| Outcome: | Extensive experiments on public benchmarks and an in-house e-commerce dataset demonstrate Unilogit’s superior performance in balancing forget and retain objectives. |
ClusComp: A Simple Paradigm for Model Compression and Efficient Finetuning (2025.findings-acl)
Copied to clipboard
| Challenge: | Weight-only quantization reduces model size but suffers from performance degradation at lower bit widths. |
| Approach: | They propose a weight-only quantization paradigm that clusters weight matrices into codebooks and finetunes them block-by-block. |
| Outcome: | The proposed paradigm outperforms quantization methods and fine tunes LLMs to 1-bit compression and fine tuning. |