Papers by Aditya Kusupati
SHARCS: Efficient Transformers Through Routing with Dynamic Width Sub-networks (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Several methods have been proposed to improve the inference efficiency of transformer-based models. |
| Approach: | They propose a new adaptive inference method that takes into account the hardness of input samples. |
| Outcome: | The proposed model outperforms or complements existing per-sample adaptive inference methods in terms of accuracy vs. FLOPs and can be applied to compressed and efficient transformer encoders to further improve their efficiency. |