Papers by Niraj Jha
DynaMo: Accelerating Language Model Inference with Dynamic Multi-Token Sampling (2024.naacl-long)
Copied to clipboard
| Challenge: | Rapid explosion in model sizes has resulted in high inference times . open-source LLMs are democratizing research in natural language processing . |
| Approach: | They propose a suite of multi-token prediction language models that reduce net inference times by leveraging traditional autoregressive weights. |
| Outcome: | The proposed model achieves same-quality generated text as baseline (Pythia-6.9B) with only 5.87% and 2.67% parameter and training time overheads. |