Papers by Kyenghun Lee
On-device System of Compositional Multi-tasking in Large Language Models (2025.emnlp-industry)
Copied to clipboard
Ondrej Bohdal, Konstantinos Theodosiadis, Asterios Mpatziakas, Dimitrios Filippidis, Iro Spyrou, Christos Zonios, Anastasios Drosou, Dimosthenis Ioannidis, Kyenghun Lee, Jijoong Moon, Hyeonmok Ko, Mete Ozay, Umberto Michieli
| Challenge: | Existing approaches to generative AI for large language models struggle when executing complex tasks simultaneously. |
| Approach: | They propose a novel approach tailored specifically for compositional multi-tasking scenarios . they add a learnable projection layer on top of the combined summarization and translation adapters. |
| Outcome: | The proposed approach performs well and is fast in both cloud-based and on-device implementations. |
HydraOpt: Navigating the Efficiency-Performance Trade-off of Adapter Merging (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods that produce a fixed trade-off between storage size and performance are often ineffective due to the growing size of large language models. |
| Approach: | They propose a model merging technique that capitalizes on similarities between low-rank adapters to reduce storage costs and improve performance. |
| Outcome: | The proposed method significantly reduces storage size (48% reduction) while outperforms existing merging techniques in terms of performance (0.2-1.8% drop). |
Efficient Compositional Multi-tasking for On-device Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Adapter parameters provide a mechanism to modify the behavior of machine learning models and have gained significant popularity in the context of large language models (LLMs). |
| Approach: | They propose a benchmark for text-based compositional multi-tasking where multiple tasks are executed simultaneously. |
| Outcome: | The proposed method is optimized for on-device applications where computational resources are limited. |
Grouped Adaptive Weight Sharing (GAWS): An Inference-Efficient Adaptation Method for Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Low-Rank Adaptation (LoRA) is a new approach to fine-tuning large language models . adapters are lightweight, task specific modules that can be used for adapters in latency-sensitive settings. |
| Approach: | They propose a low-rank adapter with a weight sharing mechanism that reduces latency by 40% . they analyze LoRA adapters on GPUs and identify segmented function calls as the primary source of latency. |
| Outcome: | The proposed adapter reduces latency to about 40% of the gap between the unmerged LoRA and the base model while maintaining parameter efficiency and comparable accuracy. |