Papers by Lianshang Cai
PILE: Pairwise Iterative Logits Ensemble for Multi-Teacher Labeled Distillation (2022.emnlp-industry)
Copied to clipboard
Lianshang Cai, Linhao Zhang, Dehong Ma, Jun Fan, Daiting Shi, Yi Wu, Zhicong Cheng, Simiu Gu, Dawei Yin
| Challenge: | Pre-trained language models have been a key part of ranking systems . knowledge distillation is widely used to maintain high performance while keeping efficient computations. |
| Approach: | They propose an algorithm to combine knowledge from multi-teachers and label information to achieve competitive performance in offline and online experiments. |
| Outcome: | The proposed method has been deployed in a real-world commercial search system. |