Papers by Taha Ceritli
K-Merge: Online Continual Merging of Adapters for On-device Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are powerful general-purpose models that can be adapted to a wide range of problem types in many languages. |
| Approach: | They propose a method for on-device online continual merging to integrate new LoRAs when a new one becomes available. |
| Outcome: | The proposed approach outperforms other methods while adhering to storage budget constraints. |
HydraOpt: Navigating the Efficiency-Performance Trade-off of Adapter Merging (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods that produce a fixed trade-off between storage size and performance are often ineffective due to the growing size of large language models. |
| Approach: | They propose a model merging technique that capitalizes on similarities between low-rank adapters to reduce storage costs and improve performance. |
| Outcome: | The proposed method significantly reduces storage size (48% reduction) while outperforms existing merging techniques in terms of performance (0.2-1.8% drop). |
A Study of Parameter Efficient Fine-tuning by Learning to Efficiently Fine-Tune (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for parameter-efficient fine-tuning (PEFT) are limited due to the need for increased computational resources. |
| Approach: | They propose a method to learn PEFT parameters from data by projecting high dimensional parameters onto low dimensional parameter manifolds or identifying PEFT parametrically. |
| Outcome: | The proposed method can be used to identify PEFT parameters on GLUE tasks. |