Papers by Juan Yeo
Stable On-Policy Distillation through Adaptive Target Reformulation (2026.findings-acl)
Copied to clipboard
| Challenge: | Knowledge distillation (KD) is widely used for transferring capabilities from proprietary models to efficient open-source counterparts. |
| Approach: | They propose a method that constructs a geometric target distribution in logit space to emphasize agreement between the teacher and the student. |
| Outcome: | Experiments show that the proposed method outperforms supervised fine-tuning and existing on-policy baselines. |