Zero-shot Sharpness-Aware Quantization for Pre-trained Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing zero-shot quantization methods are based on overfitting problem in adversarial learning process, leading to sub-optimal performance. |
| Approach: | They propose a zero-shot sharpness-aware quantization framework for the quantization of various PLMs by optimizing a minimax problem. |
| Outcome: | The proposed framework can achieve significant performance gains on discriminative and generative PLMs. |
Similar Papers
PE-QAT: Parameter-Efficient Quantization-Aware Training for Large Language Models (2026.acl-srw)
Copied to clipboard
| Challenge: | Quantization Aware Training (QAT) is expensive to train and unscalable to large models. |
| Approach: | They propose a parameter-efficient framework targeting per-channel 4-bit weight-activation quantization of large language models. |
| Outcome: | The proposed framework preserves accuracy within 0.11 percentage points of the full-precision baseline on Llama-2-7B zero-shot tasks while training only 1.26% of total parameters. |
Pre-trained Language Models Can be Fully Zero-Shot Learners (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to pre-trained language models require fine-tuning on labeled datasets or manually constructing proper prompts. |
| Approach: | They propose a nonparametric prompting PLM for fully zero-shot language understanding . they compare it to previous methods for text classification and text entailment . |
| Outcome: | The proposed method outperforms previous methods on diverse tasks. |
pQuant: Towards Effective Low-Bit Language Models via Decoupled Linear Quantization-Aware Training (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for building efficient large language models with sub 2-bit weights are lacking in accuracy and scalability. |
| Approach: | They propose a method that decouples parameters by splitting linear layers into two specialized branches. |
| Outcome: | The proposed method achieves state-of-the-art performance in extremely low-bit quantization. |
Beyond Dynamic Quantization: An Efficient Static Hierarchical Mix-precision Framework for Near-Lossless LLM Compression (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existing methods for dynamic quantization are hardware-unfriendly and often lead to large quantization errors in static scenarios. |
| Approach: | They propose a Static Hierarchical Mix-precision Quantization method which quantifies both inter-layer and intra-layer sensitivity through unified derivations involving Hessian. |
| Outcome: | The proposed method achieves 75.58% on zero-shot reasoning tasks while yielding average speedup of 2.86. |
PreQuant: A Task-agnostic Quantization Approach for Pre-trained Language Models (2023.findings-acl)
Copied to clipboard
Zhuocheng Gong, Jiahao Liu, Qifan Wang, Yang Yang, Jingang Wang, Wei Wu, Yunsen Xian, Dongyan Zhao, Rui Yan
| Challenge: | Quantization is a viable solution for pre-trained language models, but most existing methods are task-specific and require customized training and quantization with a large number of trainable parameters. |
| Approach: | They propose a "quantize before fine-tuning" framework that allows for quantization with a large number of trainable parameters on each individual task. |
| Outcome: | The proposed framework is compatible with quantization-aware training and post-training quantization and corrects quantization errors. |
LLM-QAT: Data-Free Quantization Aware Training for Large Language Models (2024.findings-acl)
Copied to clipboard
Zechun Liu, Barlas Oguz, Changsheng Zhao, Ernie Chang, Pierre Stock, Yashar Mehdad, Yangyang Shi, Raghuraman Krishnamoorthi, Vikas Chandra
| Challenge: | Several post-training quantization methods have been shown to perform well down to 8-bits. |
| Approach: | They propose a data-free distillation method that leverages generations produced by the pre-trained model to quantize any generative model independent of its training data. |
| Outcome: | The proposed method outperforms SoTA PTQ and LLaMA models at low bit precision. |
PTQ1.61: Push the Real Limit of Extremely Low-Bit Post-Training Quantization Methods for Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for sub 2-bit quantization introduce an extra 1-bit or more per weight. |
| Approach: | They propose a sub 2-bit post-training quantization method that enables weight quantization to 1.61-bit for the first time. |
| Outcome: | The proposed method reduces the upper bound of quantization error to 1.61-bit for the first time. |
Compression of Generative Pre-trained Language Models via Quantization (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods to compress generative pre-trained language models fail on generative tasks due to homogeneous word embeddings and limited memory. |
| Approach: | They propose a token-level contrastive distillation method to learn distinguishable word embeddings and a module-wise dynamic scaling method to make quantizers adaptive to different modules. |
| Outcome: | The proposed method outperforms the state-of-the-art compression methods on generative PLMs by a clear margin. |
DL-QAT: Weight-Decomposed Low-Rank Quantization-Aware Training for Large Language Models (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Quantization-aware Training (QAT) is a popular technique for reducing memory usage and improving computational efficiency in large language models. |
| Approach: | They propose a weight-decomposed low-rank quantization-aware training approach that integrates QAT with a group-specific quantization magnitude adjustment. |
| Outcome: | The proposed method outperforms the state-of-the-art method on LLaMA and LLama2 models. |
AlphaTuning: Quantization-Aware Parameter-Efficient Adaptation of Large-Scale Pre-Trained Language Models (2022.findings-emnlp)
Copied to clipboard
Se Jung Kwon, Jeonghoon Kim, Jeongin Bae, Kang Min Yoo, Jin-Hwa Kim, Baeseong Park, Byeongwook Kim, Jung-Woo Ha, Nako Sung, Dongsoo Lee
| Challenge: | Existing approaches to improve inference efficiency by accelerating model fine-tuning have not been thoroughly explored. |
| Approach: | They propose to combine parameter-efficient adaptation and model compression to accelerate model . they propose to freeze binary parameters and scale scaling factors for target tasks . |
| Outcome: | The proposed algorithm achieves >10x compression ratio under 4-bit quantization and >1,000x reduction in trainable parameters. |