Papers by Federico Lebron
GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints (2023.emnlp-main)
Copied to clipboard
| Challenge: | Multi-query attention (MQA) can lead to quality degradation and training instability . it may not be feasible to train separate models optimized for quality and inference. |
| Approach: | They propose a recipe for uptraining existing multi-head language model checkpoints into models with MQA using 5% of original training compute. |
| Outcome: | The proposed model achieves comparable quality to multi-head attention with comparable speed. |