Papers by Zhuocheng Gong
Graph-Structured Speculative Decoding (2024.findings-acl)
Copied to clipboard
Zhuocheng Gong, Jiahao Liu, Ziyue Wang, Pengfei Wu, Jingang Wang, Xunliang Cai, Dongyan Zhao, Rui Yan
| Challenge: | Speculative decoding is a promising technique to accelerate the inference of Large Language Models. |
| Approach: | They propose a method that uses a token graph to record multiple sequence hypotheses within a single draft stage. |
| Outcome: | The proposed method significantly accelerates the inference of Large Language Models (LLMs) it allows the LLM to choose from and select the longest sequence that meets its standards. |
Improving Input-label Mapping with Demonstration Replay for In-context Learning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | In-context learning (ICL) is an emerging capability of large autoregressive language models where a few demonstrations are appended to the input to enhance the model’s understanding of downstream NLP tasks without directly adjusting the model parameters. |
| Approach: | They propose a method where a few demonstrations are appended to the input to enhance the model's understanding of downstream NLP tasks without directly adjusting the model parameters. |
| Outcome: | The proposed method significantly improves the input-label mapping in ICL demonstrations. |
Mixture-of-Modules: Reinventing Transformers as Dynamic Assemblies of Modules (2024.emnlp-main)
Copied to clipboard
| Challenge: | Empirical results show that MoMs consistently outperform vanilla transformers . |
| Approach: | They propose an architecture that allows for a mixture-of-modules computation that uses a finite set of modules defined by multi-head attention and feed-forward networks. |
| Outcome: | The proposed architecture outperforms vanilla Transformers and their variants in multiple ways. |
Finding the Dominant Winning Ticket in Pre-Trained Language Models (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on pre-trained language models show that they can fine-tune parameters but achieve good downstream performance. |
| Approach: | They find that a dominant winning ticket takes up 0.05% of the parameters and is transferable across different tasks. |
| Outcome: | The proposed model can achieve comparable performance with the full-parameter model, the authors show . the dominant winning ticket takes up 0.05% of the parameters, and the model is transferable across tasks, they show - the authors conclude . |
Shorten After You’re Right: Lazy Length Penalties for Reasoning RL (2026.findings-acl)
Copied to clipboard
Danlong Yuan, Tian Xie, Shaohan Huang, Huishuai Zhang, Zhuocheng Gong, Chong Luo, Furu Wei, Dongyan Zhao
| Challenge: | Existing shortening methods for long reasoning models rely on additional supervision or multi-stage post-training. |
| Approach: | They propose a lazy length penalty that imposes length pressure on models without extra training stages. |
| Outcome: | The proposed method significantly reduces response length without extra training stages while maintaining or improving performance. |
PreQuant: A Task-agnostic Quantization Approach for Pre-trained Language Models (2023.findings-acl)
Copied to clipboard
Zhuocheng Gong, Jiahao Liu, Qifan Wang, Yang Yang, Jingang Wang, Wei Wu, Yunsen Xian, Dongyan Zhao, Rui Yan
| Challenge: | Quantization is a viable solution for pre-trained language models, but most existing methods are task-specific and require customized training and quantization with a large number of trainable parameters. |
| Approach: | They propose a "quantize before fine-tuning" framework that allows for quantization with a large number of trainable parameters on each individual task. |
| Outcome: | The proposed framework is compatible with quantization-aware training and post-training quantization and corrects quantization errors. |