Papers by Chin-Lun Fu
AdapterBias: Parameter-efficient Token-dependent Representation Shift for Adapters in NLP Tasks (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to train transformers with millions of parameters require large storage. |
| Approach: | They propose a transformer-based adapter architecture that adds a token-dependent shift to the hidden output of transformer layers to adapt to downstream tasks with only a vector and a linear layer. |
| Outcome: | The proposed model significantly reduces trainable parameters with minimal performance loss compared to fine-tuned models. |