Papers by Yuting Gao
Compensate Quantization Errors: Make Weights Hierarchical to Compensate Each Other (2024.findings-naacl)
Copied to clipboard
| Challenge: | Emergent Large Language Models (LLMs) use extraordinary performance and powerful deduction capacity to discern from traditional language models. |
| Approach: | They propose a method that uses weights to compensate quantization error and learnable singular value incremental (LSI) LSI is a technique that helps weights compensate each other conditioned on activation. |
| Outcome: | The proposed method achieves state-of-the-art performance in diverse quantization settings, no matter in weight-only, weight-activation or extremely low bit scenarios. |
Sinkhorn Distance Minimization for Knowledge Distillation (2024.lrec-main)
Copied to clipboard
Xiao Cui, Yulei Qin, Yuting Gao, Enwei Zhang, Zihan Xu, Tong Wu, Ke Li, Xing Sun, Wengang Zhou, Houqiang Li
| Challenge: | Existing knowledge distillation methods investigate divergence measures but fail to deliver effective supervision when few distribution overlap exists between teacher and student. |
| Approach: | They propose a knowledge distillation method that exploits the Sinkhorn distance to ensure a nuanced assessment of the disparity between teacher and student distributions. |
| Outcome: | The proposed method outperforms state-of-the-art methods on all kinds of LLMs with encoder-only, encoder decoder, and decoded architectures. |
Harnessing the Power of Large Language Model for Uncertainty Aware Graph Processing (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for graph processing rely on assumptions about data relations that are inadequate when handling large and complex graph data. |
| Approach: | They propose a large language model enhanced by an uncertainty-aware module to provide a confidence score on the generated graph data. |
| Outcome: | The proposed approach surpasses state-of-the-art algorithms by a substantial margin on ten datasets. |