QUANTIZATION

Binary and Ternary Natural Language Generation (2023.acl-long)

Challenge:	a tutorial on scaling natural language processing will recapitulate the state-of-the-art in the field .
Approach:	This cutting-edge tutorial recapitulates the state-of-the-art in natural language processing with scale in perspective.
Outcome:	This cutting-edge tutorial recapitulates the state-of-the-art in natural language processing with scale in perspective.

Copied to clipboard

Zechun Liu, Barlas Oguz, Aasish Pappu, Yangyang Shi, Raghuraman Krishnamoorthi

Text Generation

Challenge:	ternary and binary neural networks have proven difficult to optimize since both parameter and output space are discretized . authors demonstrate ternaries and binary models on downstream tasks of summarization and machine translation .
Approach:	They propose to use ternary and binary neural networks to optimize for multiplication-free computation . they propose to apply statistics-based quantization for the weights and elastic quantization of the activations to the transformer text generation model.
Outcome:	The proposed model outperforms the best existing models on machine translation tasks.

Domain-specific transformer models for query translation (2023.acl-industry)

Copied to clipboard

Mandar Kulkarni, Nikesh Garera, Anusua Trivedi

LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit (2024.emnlp-industry)

Challenge:	In domains such as Grocery, users prefer to buy certain brands of products . a large non-English speaking population makes it difficult to translate code-mix queries .
Approach:	They propose a model to preserve/correct Grocery brand names while translating context words . they propose to use a dataset of popular Groceries brand names to train the model .
Outcome:	The proposed model preserves/corrects Grocery brand names while translating context words . it is tested with a large non-English speaking population and is deployed in production .

Copied to clipboard

Ruihao Gong, Yang Yong, Shiqiao Gu, Yushi Huang, Chengtao Lv, Yunchen Zhang, Dacheng Tao, Xianglong Liu

Gender bias amplification during Speed-Quality optimization in Neural Machine Translation (2021.acl-short)

Challenge:	Existing quantization techniques have been categorized as 'simple' and 'highly efficient' however, their configurations vary from each other and cannot be fairly compared .
Approach:	They propose a plug-and-play compression toolkit to explore the impact of quantization.
Outcome:	The proposed toolkit explores the impact of quantization on large language models.

Copied to clipboard

Adithya Renduchintala, Denise Diaz, Kenneth Heafield, Xian Li, Mona Diab

Multilinguality

Challenge:	et al., 2002) show that gendered noun translation performance degrades faster than BLEU.
Approach:	They propose to use greedy search, quantization, AANs and shallow decoders to speed up decoding . they find minimal degradation of BLEU, but gendered noun translation degrades faster .
Outcome:	The proposed model degrades gendered noun translation performance faster than other models.

How Quantization Shapes Bias in Large Language Models (2026.eacl-long)

Copied to clipboard

Federico Marcuzzi, Xuefei Ning, Roy Schwartz, Iryna Gurevych

Proofread: Fixes All Errors with One Tap (2024.acl-demos)

Challenge:	a systematic review of quantization's effects on model biases focuses on stereotypes, fairness, toxicity, and sentiment.
Approach:	They focus on weight and activation quantization strategies and examine their effects across bias types including stereotypes, fairness, toxicity, and sentiment.
Outcome:	The proposed method can reduce stereotypes and unfairness, but it tends to increase stereotypes in generative tasks.

Copied to clipboard

Renjie Liu, Yanxiang Zhang, Yun Zhu, Haicheng Sun, Yuanbo Zhang, Michael Huang, Shanqing Cai, Lei Meng, Shumin Zhai

MobileNMT: Enabling Translation in 15MB and 30ms (2023.acl-industry)

Challenge:	Extensive experiments on a human-labeled golden set showed our tuned PaLM2-XS model achieved 85.56% good ratio.
Approach:	They propose a two-stage tuning approach to acquire the dedicated Large Language Model for the feature, followed by a reinforcement learning approach for targeted refinement.
Outcome:	The proposed model achieves 85.56% good quality on Rewrite and proofread tasks on human-labeled golden sets.

Copied to clipboard

Ye Lin, Xiaohui Wang, Zhexi Zhang, Mingxuan Wang, Tong Xiao, Jingbo Zhu

Multilinguality

Challenge:	Existing work on NMT models is limited in storage, memory, computation and power consumption.
Approach:	They propose a mobile machine translation system that can translate in 15MB and 30ms on devices.
Outcome:	The proposed system can translate in 15MB and 30ms on mobile devices.

BMCook: A Task-agnostic Compression Toolkit for Big Models (2022.emnlp-demos)

Copied to clipboard

Zhengyan Zhang, Baitao Gong, Yingfa Chen, Xu Han, Guoyang Zeng, Weilin Zhao, Yanxu Chen, Zhiyuan Liu, Maosong Sun

Can Post-Training Quantization Benefit from an Additional QLoRA Integration? (2025.naacl-industry)

Challenge:	Existing efforts to compress medium-sized models for specific tasks have limited results.
Approach:	They propose a task-agnostic compression toolkit for big models that implements quantization, pruning, distillation and MoEfication methods.
Outcome:	The proposed tool improves performance on a model with 3 billion parameters by 12x . it also outperforms the original model on three typical NLP benchmarks.

Copied to clipboard

Xiliang Zhu, Elena Khasanova, Cheng Chen

Teacher Intervention: Improving Convergence of Quantization Aware Training for Ultra-Low Precision Transformers (2023.eacl-main)

Challenge:	Large language models require considerable computing resources, which can be costly and often unavailable.
Approach:	They propose to integrate 4-bit Post-training Quantization with QLoRA to address these issues . they demonstrate that the integration outperforms standard quantization and fine-tuning .
Outcome:	The proposed integration outperforms standard PTQ and 16-bit full-parameter fine-tuning on LLMs.

Copied to clipboard

Minsoo Kim, Kyuhong Shim, Seongmin Park, Wonyong Sung, Jungwook Choi

From 2:4 to 8:16 sparsity patterns in LLMs for Outliers and Weights with Variance Correction (2026.acl-industry)

Challenge:	Quantization-aware training (QAT) is a promising method to lower the implementation cost and energy consumption.
Approach:	They propose a method for fast converging QAT of pre-trained Transformers using a layer-wise signal propagation method with the intact signal from the teacher.
Outcome:	The proposed method achieves superior accuracy with significantly lower fine-tuning iterations on well-known Transformers of natural language processing as well as computer vision compared to the state-of-the-art methods.

Copied to clipboard

Egor Maximov, Yulia Kuzkina, Egor Shvetsov, Azamat Kanametov, Aleksandr Prutko, Maxim Zhelnin, Aleksei Goncharov

LLMs on a Budget? Say HOLA (2025.emnlp-industry)

Challenge:	Quantization and sparsification are important for large language models, but they struggle to meet performance thresholds due to limited flexibility and sensitivity to outlier weights.
Approach:	They propose to use 8:16 semi-structured sparsity to surpass performance thresholds . they also show that structured sparsification for outliers is competitive with unstructured approaches .
Outcome:	The proposed method surpasses the Performance Threshold, compared to 2:4 sparsity, and offers greater flexibility with minimal storage overhead.

Copied to clipboard

Zohaib Hasan Siddiqui, Jiechao Gao, Ebad Shabbir, Mohammad Anas Azeez, Rafiq Ali, Gautam Siddharth Kashyap, Usman Naseem

Information Retrieval

Challenge:	Current solutions such as quantization, pruning, and Retrieval-Augmented Generation (RAG) offer only partial optimizations and often sacrifice accuracy, speed, or generality.
Approach:	They propose an end-to-end optimization framework for efficient LLM deployment . it leverages Hierarchical Speculative Decoding (HSD) for faster inference without quality loss.
Outcome:	HOLA delivers +17.6% EMA on GSM8K, +10.5% MCA on ARC, and reduced latency and memory on edge devices like Jetson Nano.

LongSpec: Long-Context Lossless Speculative Decoding with Efficient Drafting and Verification (2026.acl-long)

Copied to clipboard

Penghui Yang, Cunxiao Du, Fengzhuo Zhang, Haonan Wang, Tianyu Pang, Chao Du, Bo An

The Impact of Inference Acceleration on Bias of LLMs (2025.naacl-long)

Challenge:	Large Language Models (LLMs) can process extremely long contexts, requiring efficient inference over extended inputs.
Approach:	They propose a model that uses a constant-sized key-value cache to train long-context models.
Outcome:	Experimental results show that LongSpec achieves 3.26x speedup over strong Flash Attention baselines and 2.34x wall clock time on four math reasoning tasks.

Copied to clipboard

Elisabeth Kirsten, Ivan Habernal, Vedant Nanda, Muhammad Bilal Zafar

Challenge:	Recent work suggests strategies to increase inference efficiency with LLMs . however, these strategies may inadvertently lead to some side-effects.
Approach:	They propose to optimize inference acceleration strategies such as quantization, pruning, and caching to reduce inference cost and latency while maintaining predictive performance.
Outcome:	The proposed strategies reduce cost and latency while maintaining predictive performance while preserving the model size.

L4Q: Parameter Efficient Quantization-Aware Fine-Tuning on Large Language Models (2025.acl-long)

Copied to clipboard

Hyesung Jeon, Yulhwa Kim, Jae-Joon Kim

Outlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling (2023.emnlp-main)

Challenge:	Quantization-aware PEFT methods have been developed to reduce memory and computational costs associated with large language models.
Approach:	They propose a method that integrates Quantization-Aware Training (QAT) with LoRA to reduce memory overhead and improve model accuracy.
Outcome:	The proposed method significantly reduces QAT’s memory overhead while preserving the advantage of QAT in producing fully quantized LLMs with high accuracy.

Copied to clipboard

Xiuying Wei, Yunchen Zhang, Yuhang Li, Xiangguo Zhang, Ruihao Gong, Jinyang Guo, Xianglong Liu

Scaling Down, Serving Fast: Compressing and Deploying Efficient LLMs for Recommendation Systems (2025.emnlp-industry)

Challenge:	asymmetric outliers in transformer language models are a challenge for post-training quantization . we propose a framework for outlier suppression that can be seamlessly migrated into subsequent modules .
Approach:	They propose a framework for post-training quantization that includes the channel-wise shifting and scaling for concentration.
Outcome:	The proposed framework can be migrated into subsequent modules while maintaining equivalence.

Copied to clipboard

Kayhan Behdin, Ata Fatahibaarzi, Qingquan Song, Yun Dai, Aman Gupta, Zhipeng Wang, Hejian Sang, Shao Tang, Gregory Dexter, Sirou Zhu, Siyu Zhu, Tejas Dharamsi, Vignesh Kothapalli, Zhoutong Fu, Yihan Cao, Pin-Lun Hsu, Fedor Borisyuk, Natesh S. Pillai, Luke Simon, Rahul Mazumder

When Quantization Affects Confidence of Large Language Models? (2024.findings-naacl)

Challenge:	Large language models (LLMs) have demonstrated remarkable performance across a wide range of industrial applications.
Approach:	They propose two techniques for training and deploying small language models that deliver high performance for a variety of industry use cases.
Outcome:	The proposed techniques retain much of the quality of larger models while reducing training/serving costs and latency.

Copied to clipboard

Irina Proskurina, Luc Brun, Guillaume Metzler, Julien Velcin

Recover-LoRA: Data-Free Accuracy Recovery of Degraded Language Models via Low-Rank Adaptation (2025.emnlp-industry)

Challenge:	Existing studies have shown that quantization compromises performance and exacerbates biases in Large Language Models.
Approach:	They propose an explanation for quantization loss based on confidence levels . they propose a range of efficient compression and acceleration methods including quan-tization .
Outcome:	The proposed methods show that quantization decreases confidence regarding true labels and that it exacerbates biases across different scales.

Copied to clipboard

Devleena Das, Rajeev Patwari, Ashish Sirasao

Compensate Quantization Errors: Make Weights Hierarchical to Compensate Each Other (2024.findings-naacl)

Challenge:	Inference optimizations such as quantization, pruning, format and datatype conversion, model export, and serialization can lead to functional degradations in language model task performance.
Approach:	They propose a lightweight and dataset-agnostic method to recover model accuracies from quantization, pruning, format and datatype conversion, model export, and serialization errors.
Outcome:	The proposed method recovers model accuracies by 5-17% on MHA and GQA models.

Copied to clipboard

Yifei Gao, Jie Ou, Lei Wang, Yuting Xiao, Xiangzhiyuan Xiangzhiyuan, Ruiting Dai, Jun Cheng

Operator Selection and Ordering in a Pipeline Approach to Efficiency Optimizations for Transformers (2023.findings-acl)

Challenge:	Emergent Large Language Models (LLMs) use extraordinary performance and powerful deduction capacity to discern from traditional language models.
Approach:	They propose a method that uses weights to compensate quantization error and learnable singular value incremental (LSI) LSI is a technique that helps weights compensate each other conditioned on activation.
Outcome:	The proposed method achieves state-of-the-art performance in diverse quantization settings, no matter in weight-only, weight-activation or extremely low bit scenarios.

Copied to clipboard

Ji Xin, Raphael Tang, Zhiying Jiang, Yaoliang Yu, Jimmy Lin

Quadapter: Adapter for GPT-2 Quantization (2022.findings-emnlp)

Challenge:	Natural language processing tasks rely on complex neural models . transformer-based models are typically slow to execute, making it a non-trivial challenge to apply them in real-world applications.
Approach:	They propose to consider an efficiency method as an operator applied on a model . they find that the commutativity and cumulativeness of efficiency operators are plausible .
Outcome:	The proposed method is commutative and cumulative, and the results are estimated by combining methods.

Copied to clipboard

Minseop Park, Jaeseong You, Markus Nagel, Simyung Chang

Towards Fast and Accurate Neural Chinese Word Segmentation with Multi-Criteria Learning (2020.coling-main)

Challenge:	Quantization-aware training (QAT) fine-tunes model parameters in response to the numerical error arising from quantization.
Approach:	They propose a quantization adapter that scales activations channel-wise to make activations quantization-friendly by scaling them channel- wise.
Outcome:	The proposed method prevents overfitting and improves quantization performance.

Copied to clipboard

Weipeng Huang, Xingyi Cheng, Kunlong Chen, Taifeng Wang, Wei Chu

HqeKV: Towards Hybrid Quantization and Eviction for KV Cache in Long-Context LLM Inference (2026.findings-acl)

Challenge:	Chinese word segmentation datasets have ambiguous annotation criteria resulting in multi-grained compounds.
Approach:	They propose a domain adaptive segmenter to exploit diverse annotation criteria of datasets . they use bidirectional encoder representations from transformers to introduce open-domain knowledge .
Outcome:	The proposed model outperforms the state-of-the-art models on 10 Chinese word datasets with superior efficiency.

Copied to clipboard

He Wang, Yu Gu, Fangfang Li, Zhigang Wang, Zhenghao Liu, Ning Wang, Xiaohua Li, Ge Yu

Challenge:	autoregressive inference requires repeated computation across transformer layers.
Approach:	They propose a hybrid compression framework built on both quantization and eviction . they propose varying importance metric and flexible conversion policies to reduce memory overhead .
Outcome:	The proposed framework outperforms state-of-the-art methods under memory constraints.

MeMoTune: A Measure and Moment-Driven Fine-Tuning Framework for Quantized Large Language Models (2025.findings-acl)

Copied to clipboard

Yun Zhang, Xue Geng, Lizi Liao, Jintong Sun, Minghe Yu, Ge Yu

PTQ1.61: Push the Real Limit of Extremely Low-Bit Post-Training Quantization Methods for Large Language Models (2025.acl-long)

Challenge:	Existing methods combine quantization with parameter-efficient fine-tuning but fail to meet practical performance requirements.
Approach:	They propose a measure and moment approach to optimize objective function for superior fine-tuning results by scaling the update process through a gradient.
Outcome:	The proposed framework outperforms state-of-the-art methods on tasks like text generation, summarization, and understanding.

Copied to clipboard

Jiaqi Zhao, Miao Zhang, Ming Wang, Yuzhang Shang, Kaihao Zhang, Weili Guan, Yaowei Wang, Min Zhang

Combining Compressions for Multiplicative Size Scaling on Natural Language Tasks (2022.coling-1)

Challenge:	Existing methods for sub 2-bit quantization introduce an extra 1-bit or more per weight.
Approach:	They propose a sub 2-bit post-training quantization method that enables weight quantization to 1.61-bit for the first time.
Outcome:	The proposed method reduces the upper bound of quantization error to 1.61-bit for the first time.

Copied to clipboard

Rajiv Movva, Jinhao Lei, Shayne Longpre, Ajay Gupta, Chris DuBois

QuZO: Quantized Zeroth-Order Fine-Tuning for Large Language Models (2025.emnlp-main)

Challenge:	Quantization, knowledge distillation, and magnitude pruning are among the most popular methods for neural network compression in NLP.
Approach:	They compare accuracy vs. model size tradeoffs using quantization and distillation methods . they find that pruning provides greater benefit than quantization .
Outcome:	The proposed methods reduce model size and can accelerate inference, but their relative benefit and combinatorial interactions have not been rigorously studied.

Copied to clipboard

Jiajun Zhou, Yifan Yang, Kai Zhen, Ziyue Liu, Yequan Zhao, Ershad Banijamali, Athanasios Mouchtaris, Ngai Wong, Zheng Zhang

Quaff: Quantized Parameter-Efficient Fine-Tuning under Outlier Spatial Stability Hypothesis (2025.acl-long)

Challenge:	Large Language Models (LLMs) are quantized to lower precision to reduce memory cost and latency in inference.
Approach:	They propose a quantized zeroth-order framework for fine-tuning Large Language Models (LLMs) using low-precision forward passes.
Outcome:	The proposed method achieves comparable results to first-order methods in FP8 and superior accuracy in INT8 and INT4 training.

Copied to clipboard

Hong Huang, Dapeng Wu

Compression of Generative Pre-trained Language Models via Quantization (2022.acl-long)

Challenge:	Existing methods for quantized fine-tuning fail to address activation outliers . existing methods incur high computational/memory costs or fail to adequately address outlier activation .
Approach:	They propose a Quantized parameter-efficient fine-tuning framework that suppresses outliers exclusively in invariant channels using lightweight operations.
Outcome:	The proposed framework reduces outliers in invariant channels while reducing quantization errors.

Copied to clipboard

Chaofan Tao, Lu Hou, Wei Zhang, Lifeng Shang, Xin Jiang, Qun Liu, Ping Luo, Ngai Wong

BinaryBERT: Pushing the Limit of BERT Quantization (2021.acl-long)

Challenge:	Existing methods to compress generative pre-trained language models fail on generative tasks due to homogeneous word embeddings and limited memory.
Approach:	They propose a token-level contrastive distillation method to learn distinguishable word embeddings and a module-wise dynamic scaling method to make quantizers adaptive to different modules.
Outcome:	The proposed method outperforms the state-of-the-art compression methods on generative PLMs by a clear margin.

Copied to clipboard

Haoli Bai, Wei Zhang, Lu Hou, Lifeng Shang, Jin Jin, Xin Jiang, Qun Liu, Michael Lyu, Irwin King

The Cost of Compression: Investigating the Impact of Compression on Parametric Knowledge in Language Models (2023.findings-emnlp)

Challenge:	Recent pre-trained language models have achieved remarkable performance improvement in various tasks, but the improvement generally comes at the cost of increasing model size and computation.
Approach:	They propose a binary quantization technique which initializes binaryBERT by splitting from a ternary network.
Outcome:	The proposed model achieves state-of-the-art performance on the GLUE and SQUAD benchmarks while being 24x smaller.

Copied to clipboard

Satya Sai Srinath Namburi, Makesh Sreedhar, Srinath Srinivasan, Frederic Sala

Bag of Tricks for Optimizing Transformer Efficiency (2021.findings-emnlp)

Challenge:	Existing research on LLM compression focuses on general metrics like perplexity or downstream task accuracy.
Approach:	They propose to quantify the effect of pruning and quantization on model quality . they use the LAMA and LM-Harness benchmarks to quantify compression techniques .
Outcome:	The proposed compression techniques provide faster inference, smaller memory footprints, and enables local deployment.

Copied to clipboard

Ye Lin, Yanyang Li, Tong Xiao, Jingbo Zhu

Train Flat, Then Compress: Sharpness-Aware Minimization Learns More Compressible Models (2022.findings-emnlp)

Challenge:	Improving Transformer efficiency has become increasingly attractive in recent years.
Approach:	They propose to combine pruning, quantization, new architectures and training strategies to improve Transformer efficiency.
Outcome:	The proposed methods improve the inference efficiency of a strong Transformer system by 3.80x on CPU and 2.52x on GPU.

Copied to clipboard

Clara Na, Sanket Vaibhav Mehta, Emma Strubell

Block-wise Word Embedding Compression Revisited: Better Weighting and Structuring (2021.findings-emnlp)

Challenge:	Recent advances in hardware, modeling, and optimization for deep neural networks have led to improvements in memory and inference efficiency.
Approach:	They propose to combine sharpness-aware minimization with various model compression methods to improve model compressibility.
Outcome:	Empirically, optimizing for flatter minima leads to greater compressibility of parameters compared to vanilla Adam when fine-tuning BERT models, with little to no loss in accuracy on the GLUE text classification and SQuAD question answering benchmarks.

Copied to clipboard

Jong-Ryul Lee, Yong-Ju Lee, Yong-Hyuk Moon

Divergent Token Metrics: Measuring degradation to prune away LLM components – and optimize quantization (2024.naacl-long)

Challenge:	Existing methods for word embedding compression are limited . word embeds have a considerable size and need to be compressed to deploy on edge devices .
Approach:	They propose a block-wise low-rank approximation method for word embedding called GroupReduce . they propose 'frequency-inverse document frequency method' and a differentiable method for weighting .
Outcome:	The proposed algorithm more effectively finds word weights than competitors in most cases.

Copied to clipboard

Björn Deiseroth, Max Meuer, Nikolas Gritsch, Constantin Eichenberg, Patrick Schramowski, Matthias Aßenmacher, Kristian Kersting

Challenge:	Large Language Models (LLMs) have reshaped natural language processing with impressive capabilities, but their ever-increasing size has raised concerns about their effective deployment and the need for LLM compression.
Approach:	This study introduces the Divergent Token Metrics (DTMs) that measure token divergences that allow deeper insights into the subtleties of model compression.
Outcome:	The proposed measures can identify outliers and improve performance in the sparseness of the LLMs.

AffectCodec: Emotion-Preserving Neural Speech Codec for Expressive Speech Modeling (2026.findings-acl)

Copied to clipboard

Jiacheng Shi, Hongfei Du, Xinyuan Song, Y. Alicia Hong, Yanfu Zhang, Ashley Gao

Multimodality

Challenge:	Existing codecs optimize acoustic reconstruction, leaving emotion expressiveness insufficiently modeled at the representation level.
Approach:	They propose an emotion-guided neural speech codec that preserves emotional information while maintaining semantic fidelity and prosodic naturalness.
Outcome:	The proposed codec preserves emotional cues while maintaining semantic fidelity and prosodic naturalness.

Do Emergent Abilities Exist in Quantized Large Language Models: An Empirical Study (2024.lrec-main)

Copied to clipboard

Peiyu Liu, Zikang Liu, Ze-Feng Gao, Dawei Gao, Wayne Xin Zhao, Yaliang Li, Bolin Ding, Ji-Rong Wen

A Thorough Examination of Decoding Methods in the Era of LLMs (2024.emnlp-main)

Challenge:	Large Language Models (LLMs) require significant computational resources for deployment and use.
Approach:	They propose to use low-bit quantization methods to reduce memory footprint and increase inference rate to improve performance of Large Language Models.
Outcome:	The proposed methods can reduce the memory footprint and increase the inference rate of LLMs.

Copied to clipboard

Chufan Shi, Haoran Yang, Deng Cai, Zhisong Zhang, Yifan Wang, Yujiu Yang, Wai Lam

MoQAE: Mixed-Precision Quantization for Long-Context LLM Inference via Mixture of Quantization-Aware Experts (2025.acl-long)

Challenge:	Decoding methods are essential for converting language models from next-token predictors into practical task solvers.
Approach:	They propose to evaluate decoding methods in general-purpose large language models . they find that decoding method performance is notably task-dependent .
Outcome:	The proposed methods perform task-dependently and are influenced by alignment, model size, and quantization.

Copied to clipboard

Wei Tao, Haocheng Lu, Xiaoyang Qu, Bin Zhang, Kai Lu, Jiguang Wan, Jianzong Wang

Outlier-Safe Pre-Training for Robust 4-Bit Quantization of Large Language Models (2025.acl-long)

Challenge:	Existing approaches to optimize large language models for long-context inference are inefficient and consume memory.
Approach:	They propose a mixed-precision quantization method via mixture of experts that inputs tokens into router chunk by chunk to reduce inference overhead.
Outcome:	The proposed method outperforms state-of-the-art KV cache quantization methods on multiple benchmark datasets.

Copied to clipboard

Jungwoo Park, Taewhoo Lee, Chanwoong Yoon, Hyeon Hwang, Jaewoo Kang

Responsible & Trustworthy NLP

Challenge:	Quantization is a practical solution for deploying Large Language Models in resource-constrained environments.
Approach:	They propose an outlier-safe pre-training approach that prevents outlier formation . they validate a 1.4B-parameter model on 1 trillion tokens with no outliers .
Outcome:	The proposed model achieves a 35.7 average score on 1 trillion tokens with 2% training overhead.

Accurate KV Cache Quantization with Outlier Tokens Tracing (2025.acl-long)

Copied to clipboard

Yi Su, Yuechi Zhou, Quantong Qiu, Juntao Li, Qingrong Xia, Ping Li, Xinyu Duan, Zhefeng Wang, Min Zhang

Enhancing Model Privacy in Federated Learning with Random Masking and Quantization (2025.findings-emnlp)

Challenge:	Large Language Models (LLMs) require substantial computational resources during deployment.
Approach:	They propose a method to identify outlier tokens and exclude them from quantization . they find that the method can deliver a 6.4 times reduction in memory usage and a 2.5 times increase in throughput .
Outcome:	The proposed method delivers a 6.4 times reduction in memory usage and a 2.5 times increase in throughput under 2-bit quantization.

Copied to clipboard

Zhibo Xu, Zhu JianHao, Jingwen Xu, Changze Lv, Zhenghua Wang, Zisu Huang, Xiaohua Wang, Muling Wu, Qi Qian, Xiaoqing Zheng, Xuanjing Huang

Revisiting Pruning vs Quantization for Small Language Models (2025.findings-emnlp)

Challenge:	federated learning approaches are limited by the complexity of large language models and the need for specialized expertise to protect intellectual property.
Approach:	They propose a federated learning approach that leverages random masking to obscure a subnetwork of model parameters and applies quantization to the remaining parameters.
Outcome:	The proposed approach maintains strong model performance in federated learning settings and achieves enhanced protection of model parameters compared to baseline methods.

Copied to clipboard

Zihan Zhou, Simon Kurz, Zhixue Zhao

Modular Transformers: Compressing Transformers into Modularized Layers for Flexible Efficient Inference (2023.findings-acl)

Challenge:	Compressing Small Language Models (SLMs) is particularly suited for resource-constrained devices, but their compression dynamics remain underexplored compared to Large Language Model (LLMs).
Approach:	They evaluated post-training pruning and quantization methods across six SLMs from 0.5 to 3.8B, seven languages, and seven downstream tasks.
Outcome:	The proposed methods outperform pruning and quantization on six SLMs from 0.5 to 3.8B, seven languages, and seven downstream tasks.

Copied to clipboard

Wangchunshu Zhou, Ronan Le Bras, Yejin Choi

From Text to Source: Results in Detecting Large Language Model-Generated Content (2024.lrec-main)

Challenge:	Pre-trained sequence-to-sequence models have advanced the state of the art on text generation tasks.
Approach:	They introduce a modular encoder-decoder framework for flexible sequence-to-sequence model compression.
Outcome:	The proposed framework can achieve flexible compression ratios from 1.1x to 6x with little to moderate relative performance drop.

Copied to clipboard

Wissam Antoun, Benoît Sagot, Djamé Seddah

Text Classification

Challenge:	Large Language Models (LLMs) generate human-like text, but have ethical and misuse concerns.
Approach:	They evaluate whether a classifier trained to distinguish between source and target LLMs can detect text from an LLM without further training.
Outcome:	The proposed method detects text from target LLMs without further training.

Structured Pruning for Efficient Generative Pre-trained Language Models (2023.findings-acl)

Copied to clipboard

Chaofan Tao, Lu Hou, Haoli Bai, Jiansheng Wei, Xin Jiang, Qun Liu, Ping Luo, Ngai Wong

Fast Matrix Multiplications for Lookup Table-Quantized LLMs (2024.findings-emnlp)

Challenge:	Large-scale generative Pre-trained Language Models (PLMs) are limited in their deployment in real-world applications.
Approach:	They propose to prune the feed-forward networks of generative pre-trained language models to smaller widths without designing extra operators.
Outcome:	The proposed method achieves 1.51x/6.96x inference speedup on GPU/CPU with 67% size reduction.

Copied to clipboard

Han Guo, William Brandon, Radostin Cholakov, Jonathan Ragan-Kelley, Eric Xing, Yoon Kim

Multimodality

Challenge:	generative (tokenby-token) inference is memory-bound and requires a large amount of memory to perform.
Approach:	They propose a lookup table engine for weight-quantized large language models that uses offline restructuring of the quantized weight matrix to minimize bit manipulations associated with unpacking.
Outcome:	The proposed kernel can be 2-4x faster than existing GEMM kernels while achieving performance gains of 1.5 to 2 times.

A Comprehensive Evaluation of Quantization Strategies for Large Language Models (2024.findings-acl)

Copied to clipboard

Renren Jin, Jiangcun Du, Wuwei Huang, Wei Liu, Jian Luan, Bin Wang, Deyi Xiong

Assigning Distinct Roles to Quantized and Low-Rank Matrices Toward Optimal Weight Decomposition (2025.findings-acl)

Challenge:	Quantization studies have focused on instruction-tuned LLMs, leaving their performance on other benchmarks unclear.
Approach:	They propose a framework to evaluate quantized large language models using four dimensions . they propose to reduce the bits needed for model weights or activations with minimal performance loss .
Outcome:	The proposed framework can retain comparable performance to non-quantized LLMs on most benchmarks.

Copied to clipboard

Yoonjun Cho, Soeun Kim, Dongjae Jeon, Kyelim Lee, Beomsoo Lee, Albert No

Challenge:	Existing joint optimization methods prioritize one component at the expense of the other, resulting in suboptimal decompositions that fail to leverage each component’s unique strengths.
Approach:	They introduce Outlier-Driven Low-Rank Initialization (ODLRI) which assigns low-rank components the specific role of capturing activation-sensitive weights.
Outcome:	Experiments on Llama2 (7B, 13B, 70B, and Mistral-7B) and Llma3-8B show that incorporating ODLRI into the joint optimization framework reduces activation-aware error, minimizes quantization scale, and improves perplexity and zero-shot accuracy in low-bit settings.

Exploring Quantization for Efficient Pre-Training of Transformer Language Models (2024.findings-emnlp)

Copied to clipboard

Kamran Chitsaz, Quentin Fournier, Goncalo Mordido, Sarath Chandar

QEFT: Quantization for Efficient Fine-Tuning of LLMs (2024.findings-emnlp)

Challenge:	Quantization has proven to be effective after pre-training and during fine-tuning, but its effects on pre-trainer performance have remained unexplored.
Approach:	They propose a linear quantization strategy to be applied during the pre-training of Transformers to improve model efficiency and stability.
Outcome:	The proposed method improves model efficiency, stability, and performance while maintaining language modeling ability.

Copied to clipboard

Changhun Lee, Jun-gyu Jin, YoungHyun Cho, Eunhyeok Park

QaRL: Rollout-Aligned Quantization-Aware RL for Fast and Stable Training under Training–Inference Mismatch (2026.findings-acl)

Challenge:	Existing methods to optimize inference and fine-tuning for large language models have failed to improve all aspects of the process.
Approach:	They propose a new technique that accelerates both inference and fine-tuning while using fewer resources.
Outcome:	The proposed technique accelerates both inference and fine-tuning while using fewer resources.

Copied to clipboard

Hao Gu, Hao Wang, Jiacheng Liu, Lujun Li, Qiyuan Zhu, Bei Liu, Binxing Xu, Lei Wang, Xintong Yang, Sida Lin, Sirui Han, Yike Guo

Challenge:	Recent work has shown that reinforcement learning with simple rule-based reward functions (RLVR) can induce emergent reasoning behaviors and yield gains in challenging domains such as math problem solving.
Approach:	They propose a rollout-alignment-quantization-aware RL which aligns training-side forward with the quantized rollout to minimize mismatch.
Outcome:	The proposed approach outperforms quantized-rollout training by +5.5 on Qwen3-30B-A3B MoE for math problems while maintaining low-bit throughput.

A Comparative Study on the Impact of Model Compression Techniques on Fairness in Language Models (2023.acl-long)

Copied to clipboard

Krithika Ramesh, Arnav Chavan, Shrey Pandit, Sunayana Sitaram

Beyond the Tip of Efficiency: Uncovering the Submerged Threats of Jailbreak Attacks in Small Language Models (2025.findings-acl)

Challenge:	Existing literature demonstrates that compressing deep learning models could affect their fairness.
Approach:	They evaluate pruned, distilled, and quantized language models to assess their fairness . they also examine the impact of using multilingual models and evaluation measures .
Outcome:	The proposed methods can reduce the fairness of language models by reducing their complexity and reducing the cost of training and deployment.

Copied to clipboard

Sibo Yi, Tianshuo Cong, Xinlei He, Qi Li, Jiaxing Song

Responsible & Trustworthy NLP

Challenge:	Small language models (SLMs) have become increasingly prominent in the deployment on edge devices due to their high efficiency and low computational cost.
Approach:	They evaluate the security performance of 13 state-of-the-art small language models under various jailbreak attacks.
Outcome:	The proposed methods demonstrate that SLMs are quite susceptible to jailbreak attacks and some are even vulnerable to harmful prompts.

Beyond Perplexity: Multi-dimensional Safety Evaluation of LLM Compression (2024.findings-emnlp)

Copied to clipboard

Zhichao Xu, Ashim Gupta, Tao Li, Oliver Bentham, Vivek Srikumar

Softpick: No Attention Sink, No Massive Activations with Rectified Softmax (2026.findings-acl)

Challenge:	Prior work on compression prioritizes preserving perplexity, which is analogous to training loss.
Approach:	They examine the impact of model compression along four dimensions: degeneration harm, representational harm, dialect bias, and language modeling and downstream task performance.
Outcome:	The proposed compression methods can lead to unexpected consequences, the authors show . quantization preserves bias while pruning degrades quickly.

Copied to clipboard

Zayd Muhammad Kawakibi Zuhri, Erland Hilman Fuadi, Alham Fikri Aji

RA-LoRA: Rank-Adaptive Parameter-Efficient Fine-Tuning for Accurate 2-bit Quantized Large Language Models (2024.findings-acl)

Challenge:	Quantized models using softpick outperform softmax on standard benchmarks . softmax is widely used in statistics and especially in machine learning .
Approach:	They introduce a rectified, not sum-to-one, drop-in replacement for softmax in transformer attention mechanisms that eliminates attention sink and massive activations.
Outcome:	The proposed model outperforms softmax on benchmarks with lower bit precisions.

Copied to clipboard

Minsoo Kim, Sihwa Lee, Wonyong Sung, Jungwook Choi

How Does Quantization Affect Multilingual LLMs? (2024.findings-emnlp)

Challenge:	Large language models (LLMs) with their extensive parameters and high memory demands are challenging to fine-tune for specific applications with limited resources.
Approach:	They propose a method that dynamically adjusts the adapter’s rank using rank-subspace analysis, optimizing performance with fewer parameters.
Outcome:	The proposed method improves model accuracy with minimal parameter changes and demonstrates the importance of rank dynamics in optimizing quantized LLMs.

Copied to clipboard

Kelly Marchisio, Saurabh Dash, Hongyu Chen, Dennis Aumiller, Ahmet Üstün, Sara Hooker, Sebastian Ruder

Multilinguality

Challenge:	Quantization is widely used to improve inference speed and deployment of large language models.
Approach:	They conduct a thorough analysis of quantized multilingual LLMs . they find language disparately affected by quantization, non-Latin script languages worst . authors urge consideration of multilingual performance as evaluation criterion for efficient models .
Outcome:	The results show that quantization has harmful effects on human evaluation . language performance is disparately affected by quantization, the authors say .

Pruning via Merging: Compressing LLMs via Manifold Alignment Based Layer Merging (2024.emnlp-main)

Copied to clipboard

Deyuan Liu, Zhanyue Qin, Hairu Wang, Zhao Yang, Zecheng Wang, Fangying Rong, Qingbin Liu, Yanchao Hao, Bo Li, Xi Chen, Cunhang Fan, Zhao Lv, Dianhui Chu, Zhiying Tu, Dianbo Sui

Preserving Fairness and Safety in Quantized LLMs Through Critical Weight Protection (2026.findings-acl)

Challenge:	Existing methods for parameter pruning fail to utilize the knowledge from pruned parameters.
Approach:	They propose a method that uses manifold learning and the Information Bottleneck measure to merge similar layers to preserve model performance.
Outcome:	The proposed method outperforms pruning methods on multiple datasets and LLMs with quantization and achieves substantial compression ratios.

Copied to clipboard

Muhammad Alif Al Hakim, Alfan Farizki Wicaksono, Fajri Koto

Responsible & Trustworthy NLP

Challenge:	Existing quantization methods focus on general metrics like perplexity or accuracy on standard benchmarks.
Approach:	They propose a method that preserves fairness- and safety-critical weights during quantization.
Outcome:	The proposed method reduces bias and safety degradation without costly retraining or alignment while maintaining trustworthiness while retaining efficiency.

Beyond Fixed-Length Calibration for Post-Training Compression of LLMs (2025.findings-emnlp)

Copied to clipboard

Jaehoon Oh, Dokwan Oh

Challenge:	a recent study has demonstrated that the sequence length of calibration data plays a crucial role in the effectiveness of post-training compression methods.
Approach:	They propose a calibration technique that applies masking along the sequence axis to normalized hidden states.
Outcome:	The proposed method improves perplexity and zero-shot downstream tasks performance.

One QuantLLM for ALL: Fine-tuning Quantized LLMs Once for Efficient Deployments (2025.acl-long)

Copied to clipboard

Ke Yi, Yuhui Xu, Heng Chang, Yuan Meng, Tong Zhang, Jia Li

ApiQ: Finetuning of 2-Bit Quantized Large Language Model (2024.emnlp-main)

Challenge:	Quantization has shown promise for Large Language Models, but current methods require lengthy training to alleviate quantization loss.
Approach:	They propose to decouple weights and incorporate Low-Rank adapters to reduce weight sharing . they validate the approach on LLaMA2 families and Mistral on downstream evaluation .
Outcome:	The proposed approach shows high performance while reducing deployment time faced with multiple scenarios.

Copied to clipboard

Baohao Liao, Christian Herold, Shahram Khadivi, Christof Monz

xCOMET-lite: Bridging the Gap Between Efficiency and Quality in Learned MT Evaluation Metrics (2024.emnlp-main)

Challenge:	Memory-efficient finetuning of large language models (LLMs) has attracted huge attention with the increasing size of LLMs due to the constraints posed by GPU memory limitations and the effectiveness of these methods compared to full finetune.
Approach:	They propose a memory-efficient finetuning framework called ApiQ to restore lost information from quantization by initializing LoRA components and quantizing weights of LLMs.
Outcome:	The proposed framework maintains the original LLM’s activation precision while mitigating error propagation from shallower into deeper layers.

Copied to clipboard

Daniil Larionov, Mikhail Seleznyov, Vasiliy Viskov, Alexander Panchenko, Steffen Eger

Break Through the Compression Bottleneck: From Theory to Practice (2026.findings-acl)

Challenge:	State-of-the-art trainable machine translation evaluation metrics rely on large encoders . this makes them computationally expensive and inaccessible to researchers with limited resources.
Approach:	They propose a method to extract knowledge stored in large encoders and a pipeline for efficient black-box distillation.
Outcome:	The proposed model surpasses COMET-22 and BLEURT-20 on the WMT22 dataset by 6.4%.

Copied to clipboard

Xiusheng Huang, Lu Wang, Yequan Wang, Jun Zhao, Kang Liu

ThinkSLM: Towards Reasoning in Small Language Models (2025.emnlp-main)

Challenge:	Existing compression methods suffer from bottleneck issues when compression ratio is increased.
Approach:	They propose a novel approach to combine low-rank decomposition and quantization methods to reduce the compression bottleneck.
Outcome:	The proposed method reduces the computational and memory overhead of existing methods while maintaining model accuracy.

Copied to clipboard

Gaurav Srivastava, Shuxiang Cao, Xuan Wang

Fast and Accurate Fisher-Guided Quantization via Efficient Kronecker Factorization (2026.acl-long)

Challenge:	Reasoning has long been viewed as an emergent property of large language models (LLMs), appearing at or above a certain scale (100B parameters).
Approach:	They propose a benchmark to evaluate the reasoning abilities of small language models (SLMs) using quantization, pruning, and distillation.
Outcome:	The proposed benchmark systematically evaluates reasoning abilities of small language models trained from scratch or derived from LLMs using quantization, pruning, and distillation.

Copied to clipboard

Viktoriia A. Chekalina, Gerasin Timofey, Andrey Kuznetsov, Evgeny Frolov

On the Editability of Delta Parameters in Post-Trained Models (2026.findings-acl)

Challenge:	Quantization has shown strong results in preserving model quality under compression, but under aggressive bit-width reductions, even quantization may require additional information to prevent performance degradation.
Approach:	They propose a Kronecker-factored approximation that captures second-order curvature information, captured by the Hessian, to achieve a 10 speedup over prior approaches.
Outcome:	The proposed method significantly accelerates the most expensive component in second-order quantization – Hessian parameterization . it achieves up to a 10 speedup over prior approaches.

Copied to clipboard

Qiaoyu Tang, Le Yu, Bowen Yu, Hongyu Lin, Keming Lu, Yaojie Lu, Xianpei Han, Le Sun