Papers by Janghwan Lee
Enhancing Computation Efficiency in Large Language Models through Weight and Activation Quantization (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are proficient in natural language processing tasks, but their deployment is limited by extensive parameter sizes and computational demands. |
| Approach: | They propose a method to enhance computational efficiency in large language models by 4-bit weight and 8-bit activation quantization. |
| Outcome: | The proposed techniques significantly boost task accuracies to levels comparable with full-precision models. |
AMXFP4: Taming Activation Outliers with Asymmetric Microscaling Floating-Point for 4-bit LLM Inference (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing research shows unsatisfactory empirical results for microscaling (MX) floating-point (FP) formats. |
| Approach: | They propose a 4-bit asymmetric FP format that handles activation outliers without calibration. |
| Outcome: | The proposed format outperforms MXFP4 by 3% on VQA and rotation-based methods by 1.6% on CSQA. |
Improving Conversational Abilities of Quantized Large Language Models via Direct Preference Alignment (2024.acl-long)
Copied to clipboard
| Challenge: | Quantization-aware direct preference optimization (QDPO) improves conversational abilities of quantized LLMs . token-flipping is a critical factor for degraded text generation quality . |
| Approach: | They propose a method that aligns quantized LLMs with their full-precision counterparts, improving conversational abilities. |
| Outcome: | The proposed method outperforms established fine-tuning techniques on two instruction-tuned LLMs in various languages and models, setting a new benchmark for conversational chatbot development. |