Papers by Simiao Li
Multi-Granularity Semantic Revision for Large Language Model Distillation (2026.acl-long)
Copied to clipboard
Xiaoyu Liu, Yun Zhang, Wei Li, Simiao Li, Xudong Huang, Hanting Chen, Yehui Tang, Jie Hu, Zhiwei Xiong, Yunhe Wang
| Challenge: | Existing methods for generating large language models rely on student-generated outputs, which introduce generation errors and misguide the distillation process. |
| Approach: | They propose a multi-granularity semantic revision method for LLM distillation that corrects errors using teacher-generated tokens and re-generates the sequence to minimize errors. |
| Outcome: | The proposed method reduces errors and misguides distillation on student models and improves consistency between teacher and student outputs. |