Papers by Baeseong Park
Extremely Low Bit Transformer Quantization for On-Device Neural Machine Translation (2020.findings-emnlp)
Copied to clipboard
Insoo Chung, Byeongwook Kim, Yoonjung Choi, Se Jung Kwon, Yongkweon Jeon, Baeseong Park, Sangha Kim, Dongsoo Lee
| Challenge: | Quantization is an effective technique to address heavy computation load and memory overhead during inference. |
| Approach: | They propose a low-bit quantization strategy to represent Transformer weights by an extremely low number of bits. |
| Outcome: | The proposed model achieves 11.8 smaller model size than baseline model, with less than -0.5 BLEU. |
AlphaTuning: Quantization-Aware Parameter-Efficient Adaptation of Large-Scale Pre-Trained Language Models (2022.findings-emnlp)
Copied to clipboard
Se Jung Kwon, Jeonghoon Kim, Jeongin Bae, Kang Min Yoo, Jin-Hwa Kim, Baeseong Park, Byeongwook Kim, Jung-Woo Ha, Nako Sung, Dongsoo Lee
| Challenge: | Existing approaches to improve inference efficiency by accelerating model fine-tuning have not been thoroughly explored. |
| Approach: | They propose to combine parameter-efficient adaptation and model compression to accelerate model . they propose to freeze binary parameters and scale scaling factors for target tasks . |
| Outcome: | The proposed algorithm achieves >10x compression ratio under 4-bit quantization and >1,000x reduction in trainable parameters. |