Papers by Yuncheng Song
Self-Evolution Knowledge Distillation for LLM-based Machine Translation (2025.coling-main)
Copied to clipboard
| Challenge: | Existing knowledge distillation strategies for large language models minimize output distributions between student and teacher models indiscriminately for each token. |
| Approach: | They propose a distillation strategy that integrates teacher and one-hot distribution of ground truth into the student distribution as prior knowledge, which promotes the distillation process. |
| Outcome: | The proposed method brings an average improvement of approximately 1.4 SacreBLEU points across four translation directions in the WMT22 test sets. |