Papers by Jerry Quinn
Pieces of Eight: 8-bit Neural Machine Translation (N18-3)
Copied to clipboard
| Challenge: | Neural machine translation models are trained using 32-bit floating point values and have improved fluency and adequacy. |
| Approach: | They propose to use 8-bit quantization to train models using 32-bit floating point values and show that 8- bit translation makes a non-negligible impact in terms of speed with no degradation in accuracy and adequacy. |
| Outcome: | The proposed method improves accuracy and accuracy without degradation in accuracy and adequacy. |
Zero-Shot Dynamic Quantization for Transformer Inference (2022.emnlp-industry)
Copied to clipboard
| Challenge: | Existing methods for quantizing models require calibration or modification of parameters . run-time inference of such large models is costly due to large computational requirements . |
| Approach: | They propose a run-time method for quantizing BERT-like models to 8-bit integers . they demonstrate that the method can be used on many NLP tasks without calibration steps . |
| Outcome: | The proposed method reduces the accuracy loss associated with quantizing BERT-like models to 8-bit integers. |