Papers by Zhengang Li
Efficient Transformer-based Large Scale Language Representations using Hardware-friendly Block Structured Pruning (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained large-scale language models have been criticized for their limited weight storage and computational speed on hardware platforms. |
| Approach: | They propose an efficient transformer-based large-scale language representation using hardware-friendly block structure pruning. |
| Outcome: | The proposed model achieves 5.0x accuracy on GLUE benchmarks and 1.79x compression rate on DistilBERT. |