Bit-by-Bit: Progressive QAT Strategy with Outlier Channel Splitting for Stable Low-Bit LLMs (2026.acl-long)
Copied to clipboard
Binxing Xu, Hao Gu, Lujun Li, Hao Wang, Bei Liu, Jiacheng Liu, Qiyuan Zhu, Xintong Yang, Chao Li, Sirui Han, Yike Guo
| Challenge: | Existing approaches to training LLMs at ultra-low precisions suffer from convergence instability and substantial training costs. |
| Approach: | They propose a progressive QAT framework with outlier channel splitting to address these issues . they use nested structure of integer quantization grids to enable a "train once, deploy any precision" paradigm . |
| Outcome: | The proposed framework outperforms baselines on both Llama2/3 and W2A16, with an 11 speedup over BF16. |
Similar Papers
PE-QAT: Parameter-Efficient Quantization-Aware Training for Large Language Models (2026.acl-srw)
Copied to clipboard
| Challenge: | Quantization Aware Training (QAT) is expensive to train and unscalable to large models. |
| Approach: | They propose a parameter-efficient framework targeting per-channel 4-bit weight-activation quantization of large language models. |
| Outcome: | The proposed framework preserves accuracy within 0.11 percentage points of the full-precision baseline on Llama-2-7B zero-shot tasks while training only 1.26% of total parameters. |
Beyond Dynamic Quantization: An Efficient Static Hierarchical Mix-precision Framework for Near-Lossless LLM Compression (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existing methods for dynamic quantization are hardware-unfriendly and often lead to large quantization errors in static scenarios. |
| Approach: | They propose a Static Hierarchical Mix-precision Quantization method which quantifies both inter-layer and intra-layer sensitivity through unified derivations involving Hessian. |
| Outcome: | The proposed method achieves 75.58% on zero-shot reasoning tasks while yielding average speedup of 2.86. |
Achieving binary weight and activation for LLMs using Post-Training Quantization (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for quantizing large language models suffer from performance degradation when weights are quantized to 1 bit. |
| Approach: | They propose a post-training quantization framework with W(1+1)A(14) configuration . they propose utilizing Hessian-aware fine-grained grouping along with an EM-based quantization scheme . |
| Outcome: | The proposed method surpasses state-of-the-art (SOTA) LLM quantization baselines on W2A4 across multiple tasks. |
BWLA: Breaking the Barrier of W1AX Post-Training Quantization for LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models have driven major progress in NLP, but memory and compute requirements hinder practical deployment. |
| Approach: | They propose a framework that preserves high accuracy while achieving 1-bit weight quantization . the orthogonal-kronecker transformation learns an orthogonale mapping via EM minimization - a new approach to quantization is proposed . |
| Outcome: | The proposed framework achieves 1-bit weight quantization with low activations with low-bit activations. |
Outlier-Safe Pre-Training for Robust 4-Bit Quantization of Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Quantization is a practical solution for deploying Large Language Models in resource-constrained environments. |
| Approach: | They propose an outlier-safe pre-training approach that prevents outlier formation . they validate a 1.4B-parameter model on 1 trillion tokens with no outliers . |
| Outcome: | The proposed model achieves a 35.7 average score on 1 trillion tokens with 2% training overhead. |
pQuant: Towards Effective Low-Bit Language Models via Decoupled Linear Quantization-Aware Training (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for building efficient large language models with sub 2-bit weights are lacking in accuracy and scalability. |
| Approach: | They propose a method that decouples parameters by splitting linear layers into two specialized branches. |
| Outcome: | The proposed method achieves state-of-the-art performance in extremely low-bit quantization. |
CoopQ: Cooperative Game Inspired Layerwise Mixed Precision Quantization for LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for large language models struggle when the average precision drops below four bits, limiting deployment on resourceconstrained devices such as mobiles, edge sensors, or standard GPUs. |
| Approach: | They propose a game-like game-inspired mixed-precision quantization method which translates these Shapley estimates into a binary quadratic optimization formulation, assigning either 2 or 4-bit precision to layers under strict memory constraints. |
| Outcome: | The proposed method reduces Perplexity by 20 – 80 % across average precisions spanning 4 bit down to 2 bit, compared to methods relying on isolated metrics. |
PTQ1.61: Push the Real Limit of Extremely Low-Bit Post-Training Quantization Methods for Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for sub 2-bit quantization introduce an extra 1-bit or more per weight. |
| Approach: | They propose a sub 2-bit post-training quantization method that enables weight quantization to 1.61-bit for the first time. |
| Outcome: | The proposed method reduces the upper bound of quantization error to 1.61-bit for the first time. |
DL-QAT: Weight-Decomposed Low-Rank Quantization-Aware Training for Large Language Models (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Quantization-aware Training (QAT) is a popular technique for reducing memory usage and improving computational efficiency in large language models. |
| Approach: | They propose a weight-decomposed low-rank quantization-aware training approach that integrates QAT with a group-specific quantization magnitude adjustment. |
| Outcome: | The proposed method outperforms the state-of-the-art method on LLaMA and LLama2 models. |
EfficientQAT: Efficient Quantization-Aware Training for Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Quantization-aware training (QAT) is a low-bit training solution that requires substantial training resources. |
| Approach: | They propose an algorithm that reduces memory consumption by low-bit representations with minimal accuracy loss. |
| Outcome: | EfficientQAT achieves 2-bit Llama-2-70B model on single GPU in 41 hours . compared to previous methods, it obtains model with less than 3 points accuracy degradation . |