Papers by Neo Eyal
Layer Duplication in LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Duplicating multihead self-attention layers once or twice yields significant performance boosts. |
| Approach: | They investigate the effect of duplicating multihead self-attention layers in large language models . |
| Outcome: | The proposed method improves performance across a range of language tasks with or without fine-tuning. |