Papers by Amir Aminifar
BEFT: Bias-Efficient Fine-Tuning of Language Models in Low-Data Regimes (2026.acl-long)
Copied to clipboard
| Challenge: | Fine-tuning bias terms of large language models (LLMs) for downstream tasks has gained a lot of attention over the past few years. |
| Approach: | They extensively evaluate bq, bk, v across a wide range of LLMs . they find that bv generally leads to higher downstream performance in low-data regimes compared to bQ and bK . |
| Outcome: | The proposed method improves performance across a wide range of LLMs spanning encoder-only and decoder-free architectures up to 6.7B parameters. |