| Challenge: | Existing quantization solutions are integer-based and struggle with bit widths below 8 bits. |
| Approach: | They propose a method for quantizing weights and activations in large language models down to 4-bit floating-point values in a post-training manner. |
| Outcome: | The proposed method outperforms existing methods on common sense zero-shot reasoning tasks by 12.7 points. |
Similar Papers
Achieving binary weight and activation for LLMs using Post-Training Quantization (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for quantizing large language models suffer from performance degradation when weights are quantized to 1 bit. |
| Approach: | They propose a post-training quantization framework with W(1+1)A(14) configuration . they propose utilizing Hessian-aware fine-grained grouping along with an EM-based quantization scheme . |
| Outcome: | The proposed method surpasses state-of-the-art (SOTA) LLM quantization baselines on W2A4 across multiple tasks. |
LRQ: Optimizing Post-Training Quantization for Large Language Models by Learning Low-Rank Weight-Scaling Matrices (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for quantizing weights and activations of large language models suffer from non-negligible accuracy drops, especially on massive multitask language understanding. |
| Approach: | They propose a weight-activation quantization method that reconstructs the outputs of an intermediate Transformer block by leveraging low-rank weight-scaling matrices. |
| Outcome: | The proposed method reduces the complexity of the weight-activation quantization techniques while achieving high throughput and reducing inference costs. |
Enhancing Computation Efficiency in Large Language Models through Weight and Activation Quantization (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are proficient in natural language processing tasks, but their deployment is limited by extensive parameter sizes and computational demands. |
| Approach: | They propose a method to enhance computational efficiency in large language models by 4-bit weight and 8-bit activation quantization. |
| Outcome: | The proposed techniques significantly boost task accuracies to levels comparable with full-precision models. |
BWLA: Breaking the Barrier of W1AX Post-Training Quantization for LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models have driven major progress in NLP, but memory and compute requirements hinder practical deployment. |
| Approach: | They propose a framework that preserves high accuracy while achieving 1-bit weight quantization . the orthogonal-kronecker transformation learns an orthogonale mapping via EM minimization - a new approach to quantization is proposed . |
| Outcome: | The proposed framework achieves 1-bit weight quantization with low activations with low-bit activations. |
Benchmarking Post-Training Quantization of Large Language Models under Microscaling Floating Point Formats (2026.acl-long)
Copied to clipboard
| Challenge: | Microscaling Floating-Point (MXFP) is a low-precision format for large language models (LLMs). |
| Approach: | They conduct systematic evaluations of PTQ under Microscaling Floating-Point (MXFP) . they find that MXFP8 consistently achieves near-lossless performance . |
| Outcome: | The proposed method achieves near-lossless performance while MXFP4 introduces substantial accuracy degradation and remains challenging. |
“Give Me BF16 or Give Me Death”? Accuracy-Performance Trade-Offs in LLM Quantization (2025.acl-long)
Copied to clipboard
| Challenge: | despite popularity of large language model quantization, there are significant accuracy-performance trade-offs associated with quantization formats. |
| Approach: | They evaluate popular quantization formats across academic benchmarks and real-world tasks . they also examine the difference in text generated by quantized models versus their uncompressed counterparts . |
| Outcome: | The proposed format is lossless across all model scales and incurs low accuracy degradation when properly tuned. |
ATQ: Activation Transformation forWeight-Activation Quantization of Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Quantization methods are available to solve the problem of high computational and storage costs for Large language models. |
| Approach: | They propose an INT8 weight-activation quantization method that can achieve lossless accuracy. |
| Outcome: | The proposed method can achieve lossless accuracy on OPT and LLaMA families. |
Revisiting Block-based Quantisation: What is Important for Sub-8-bit LLM Inference? (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing quantisation methods mainly focus on 8-bit LLMs . a lack of scaling offsets in the quantisation process limits the use of LLM inference. |
| Approach: | They propose to use block quantisations to reduce scaling offsets in Large language models . they find that the block quantizations reduce scaling only from an arithmetic perspective . |
| Outcome: | The proposed methods reduce scaling offsets solely from an arithmetic perspective without additional treatments in the computational path. |
AFPQ: Asymmetric Floating Point Quantization for LLMs (2024.findings-acl)
Copied to clipboard
| Challenge: | Low-bit weight quantization can save memory and accelerate inference. |
| Approach: | They propose asymmetric FP quantization which sets separate scales for positive and negative values. |
| Outcome: | The proposed method leads to large accuracy improvements and can be easily plugged into other quantization methods, including GPTQ and AWQ, for better performance. |
PTQ1.61: Push the Real Limit of Extremely Low-Bit Post-Training Quantization Methods for Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for sub 2-bit quantization introduce an extra 1-bit or more per weight. |
| Approach: | They propose a sub 2-bit post-training quantization method that enables weight quantization to 1.61-bit for the first time. |
| Outcome: | The proposed method reduces the upper bound of quantization error to 1.61-bit for the first time. |