Papers by Huiyin Xue
DReSD: Dense Retrieval for Speculative Decoding (2025.findings-acl)
Copied to clipboard
| Challenge: | Speculative decoding (SD) uses an efficient draft model to propose the next few tokens, which are verified by the LLM in a single forward call, reducing latency while preserving its outputs. |
| Approach: | They propose a draft model that proposes the next few tokens from a non-parametric datastore and uses a framework that uses approximate nearest neighbour search with contextualised token embeddings to retrieve the most semantically relevant sequences for SD. |
| Outcome: | The proposed framework achieves (on average) 87% higher acceptance rates, 65% longer accepted tokens and 19% faster generation speeds compared to sparse retrieval (REST). |
Pit One Against Many: Leveraging Attention-head Embeddings for Parameter-efficient Multi-head Attention (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing pre-trained language models have produced performance gains in various tasks but come with large computational requirements. |
| Approach: | They propose an alternative module that uses only a single shared projection matrix and multiple head embeddings (MHE) they demonstrate that MHE attention is substantially more memory efficient compared to alternative attention mechanisms. |
| Outcome: | The proposed model is more memory efficient compared to the current model while achieving high retention ratio on several downstream tasks. |
HashFormers: Towards Vocabulary-independent Pre-trained Transformers (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing pre-trained language models are vocabulary-dependent, mapping by default each token to its corresponding embedding. |
| Approach: | They propose a family of vocabulary-independent pre-trained transformers that support unlimited vocabulary . they propose to map each token to its corresponding embedding by default . |
| Outcome: | The proposed models are more memory efficient than existing models while achieving comparable performance on multiple text classification tasks. |