The Cost of Compression: Investigating the Impact of Compression on Parametric Knowledge in Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing research on LLM compression focuses on general metrics like perplexity or downstream task accuracy. |
| Approach: | They propose to quantify the effect of pruning and quantization on model quality . they use the LAMA and LM-Harness benchmarks to quantify compression techniques . |
| Outcome: | The proposed compression techniques provide faster inference, smaller memory footprints, and enables local deployment. |
Similar Papers
When Compression Meets Model Compression: Memory-Efficient Double Compression for Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) exhibit excellent performance in various tasks, but memory requirements present a challenge when deploying on memory-limited devices. |
| Approach: | They propose a framework to compress LLM after quantization further, achieving about 2.2x compression ratio. |
| Outcome: | The proposed model can achieve 40% reduction in memory size with negligible loss in accuracy and inference speed. |
Beyond Perplexity: Multi-dimensional Safety Evaluation of LLM Compression (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work on compression prioritizes preserving perplexity, which is analogous to training loss. |
| Approach: | They examine the impact of model compression along four dimensions: degeneration harm, representational harm, dialect bias, and language modeling and downstream task performance. |
| Outcome: | The proposed compression methods can lead to unexpected consequences, the authors show . quantization preserves bias while pruning degrades quickly. |
Comparing Text Compression Capabilities of Large Language Models with Traditional Compression Algorithms (2026.eacl-srw)
Copied to clipboard
| Challenge: | Experimental results show that large language models outperform baselines on non-English datasets . traditional methods remained dataset-agnostic, and the results suggest that current methods are impractical for the compression task. |
| Approach: | They evaluate the non-English and unstructured text compression performance of Large Language Models . they compare them with traditional baselines on datasets from eight most widely spoken languages . |
| Outcome: | The evaluated LLM outperformed baselines on non-English datasets . the results show that the current methods are highly impractical for the compression task . |
Understanding the Effect of Model Compression on Social Bias in Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) trained with self-supervision on vast corpora of web text fit to the social biases of that text, leading to representational harm. |
| Approach: | They propose to use quantization and knowledge distillation to reduce the computational burden of LLMs to mitigate the effects of inappropriate social biases learned during pretraining. |
| Outcome: | The proposed methods reduce the computational burden of large language models by reducing their size and complexity. |
On the Impact of Calibration Data in Post-training Quantization and Pruning (2024.acl-long)
Copied to clipboard
| Challenge: | Quantization and pruning are the foundations of compression for large language models . however, no prior work has investigated how calibration data impacts performance of compression methods. |
| Approach: | They propose an empirical study on the effect of calibration data on LLM performance. |
| Outcome: | The proposed methods improve performance in a post-training setting. |
LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit (2024.emnlp-industry)
Copied to clipboard
Ruihao Gong, Yang Yong, Shiqiao Gu, Yushi Huang, Chengtao Lv, Yunchen Zhang, Dacheng Tao, Xianglong Liu
| Challenge: | Existing quantization techniques have been categorized as 'simple' and 'highly efficient' however, their configurations vary from each other and cannot be fairly compared . |
| Approach: | They propose a plug-and-play compression toolkit to explore the impact of quantization. |
| Outcome: | The proposed toolkit explores the impact of quantization on large language models. |
LVLM-Compress-Bench: Benchmarking the Broader Impact of Large Vision-Language Model Compression (2025.findings-naacl)
Copied to clipboard
Souvik Kundu, Anahita Bhiwandiwalla, Sungduk Yu, Phillip Howard, Tiep Le, Sharath Nittur Sridhar, David Cobbley, Hao Kang, Vasudev Lal
| Challenge: | LVLMs have been shown to perform well on simple uni-modal benchmarks, but their detailed study on multi-modal models is still lacking. |
| Approach: | They propose a framework to analyze the impact of compression on LVLMs on multi-modal input driven tasks. |
| Outcome: | The proposed framework analyzes the impact of compression on generative performance of large vision language models on multi-modal input driven tasks. |
A Comparative Study on the Impact of Model Compression Techniques on Fairness in Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Existing literature demonstrates that compressing deep learning models could affect their fairness. |
| Approach: | They evaluate pruned, distilled, and quantized language models to assess their fairness . they also examine the impact of using multilingual models and evaluation measures . |
| Outcome: | The proposed methods can reduce the fairness of language models by reducing their complexity and reducing the cost of training and deployment. |
Compressing Language Models for Specialized Domains (2026.eacl-long)
Copied to clipboard
| Challenge: | Language models (LMs) excel at tasks across diverse domains, yet require substantial computational resources during inference. |
| Approach: | They propose a calibration method to improve the in-domain performance of compressed LMs in a post-training setting. |
| Outcome: | The proposed method outperforms existing methods on domain-specific tasks while preserving general performance. |
Scaling Down, Serving Fast: Compressing and Deploying Efficient LLMs for Recommendation Systems (2025.emnlp-industry)
Copied to clipboard
Kayhan Behdin, Ata Fatahibaarzi, Qingquan Song, Yun Dai, Aman Gupta, Zhipeng Wang, Hejian Sang, Shao Tang, Gregory Dexter, Sirou Zhu, Siyu Zhu, Tejas Dharamsi, Vignesh Kothapalli, Zhoutong Fu, Yihan Cao, Pin-Lun Hsu, Fedor Borisyuk, Natesh S. Pillai, Luke Simon, Rahul Mazumder
| Challenge: | Large language models (LLMs) have demonstrated remarkable performance across a wide range of industrial applications. |
| Approach: | They propose two techniques for training and deploying small language models that deliver high performance for a variety of industry use cases. |
| Outcome: | The proposed techniques retain much of the quality of larger models while reducing training/serving costs and latency. |