Papers by Jungwook Choi
Enhancing Computation Efficiency in Large Language Models through Weight and Activation Quantization (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are proficient in natural language processing tasks, but their deployment is limited by extensive parameter sizes and computational demands. |
| Approach: | They propose a method to enhance computational efficiency in large language models by 4-bit weight and 8-bit activation quantization. |
| Outcome: | The proposed techniques significantly boost task accuracies to levels comparable with full-precision models. |
RA-LoRA: Rank-Adaptive Parameter-Efficient Fine-Tuning for Accurate 2-bit Quantized Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) with their extensive parameters and high memory demands are challenging to fine-tune for specific applications with limited resources. |
| Approach: | They propose a method that dynamically adjusts the adapter’s rank using rank-subspace analysis, optimizing performance with fewer parameters. |
| Outcome: | The proposed method improves model accuracy with minimal parameter changes and demonstrates the importance of rank dynamics in optimizing quantized LLMs. |
AMXFP4: Taming Activation Outliers with Asymmetric Microscaling Floating-Point for 4-bit LLM Inference (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing research shows unsatisfactory empirical results for microscaling (MX) floating-point (FP) formats. |
| Approach: | They propose a 4-bit asymmetric FP format that handles activation outliers without calibration. |
| Outcome: | The proposed format outperforms MXFP4 by 3% on VQA and rotation-based methods by 1.6% on CSQA. |
Improving Conversational Abilities of Quantized Large Language Models via Direct Preference Alignment (2024.acl-long)
Copied to clipboard
| Challenge: | Quantization-aware direct preference optimization (QDPO) improves conversational abilities of quantized LLMs . token-flipping is a critical factor for degraded text generation quality . |
| Approach: | They propose a method that aligns quantized LLMs with their full-precision counterparts, improving conversational abilities. |
| Outcome: | The proposed method outperforms established fine-tuning techniques on two instruction-tuned LLMs in various languages and models, setting a new benchmark for conversational chatbot development. |
InfiniPot: Infinite Context Processing on Memory-Constrained LLMs (2024.emnlp-main)
Copied to clipboard
| Challenge: | InfiniPot is a KV cache control framework that can handle long input contexts without additional training. |
| Approach: | They propose a KV cache control framework that can handle long input contexts efficiently without additional training. |
| Outcome: | The proposed framework outperforms models trained for long contexts in various NLP tasks and is highly efficient and versatile. |
MapCoder-Lite: Distilling Multi-Agent Coding into a Single Small LLM (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing large-scale (> 30 B) models are costly and collapse when downsized to small open-source models. |
| Approach: | They propose a framework for distilling large, multi-agent coding systems into a single 7B model. |
| Outcome: | The proposed framework doubles xCodeEval accuracy and reduces GPU memory and token generation time by 4 compared to a 32B model. |
Understanding and Improving Knowledge Distillation for Quantization Aware Training of Large Transformer Encoders (2022.emnlp-main)
Copied to clipboard
| Challenge: | Knowledge distillation (KD) has been used for quantization-aware training to improve the ability of a lightweight model with the transferred knowledge from the teacher. |
| Approach: | They propose two methods to improve attention recovery of quantized large Transformers by combining attention-map and attention-output losses. |
| Outcome: | The proposed methods achieve state-of-the-art accuracy for quantized large Transformer encoder models with sub-2-bit weight quantization. |
Teacher Intervention: Improving Convergence of Quantization Aware Training for Ultra-Low Precision Transformers (2023.eacl-main)
Copied to clipboard
| Challenge: | Quantization-aware training (QAT) is a promising method to lower the implementation cost and energy consumption. |
| Approach: | They propose a method for fast converging QAT of pre-trained Transformers using a layer-wise signal propagation method with the intact signal from the teacher. |
| Outcome: | The proposed method achieves superior accuracy with significantly lower fine-tuning iterations on well-known Transformers of natural language processing as well as computer vision compared to the state-of-the-art methods. |