Wenru Xu, Peixuan Xu, Ziqi Yang, Ming Hu, Zihui Wang, Jianzhong Qi, Rongshan Yu, Xiaoliang Fan, Cheng Wang
| Challenge: | Speculative decoding (SD) is a training-free SD framework that orchestrates dynamic alternation combining serial dynamic drafting with parallel draft verification. |
| Approach: | They propose a serial and parallel intertwined speculative DEcoding framework that orchestrates dynamic alternation combining serial dynamic drafting and parallel draft verification. |
| Outcome: | The proposed framework accelerates inference while reducing the LLM usage costs. |
Similar Papers
Speculative Verification: Exploiting Information Gain for Speculative Decoding (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are used for many applications but their size and computational cost make inference serving a significant challenge. |
| Approach: | They propose an efficient augmentation to Speculative Decoding (SD) that predicts speculation accuracy and dynamically adapts the verification length to maximize throughput. |
| Outcome: | The proposed model reduces wasted verification on rejected tokens and improves decoding efficiency. |
Multi-Drafter Speculative Decoding with Alignment Feedback (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to accelerate large language model (LLM) inference use a smaller model to draft future tokens, which are then verified by the target LLM. |
| Approach: | They propose a unified framework that integrates multiple drafters into the SD process. |
| Outcome: | Extensive experiments show that MetaSD outperforms single-drafter approaches. |
SpecHub: Provable Acceleration to Multi-Draft Speculative Decoding (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have limited inference speed due to sequential token generation . Spechub is a novel, efficient sampling-verification method for MDSD that improves acceptance rates with only linear computational overhead. |
| Approach: | They propose a method that uses a smaller draft model to generate multiple token sequences . Spechub generates 0.05-0.27 and 0.02-0.16 more tokens per step than RRS and RRS without replacement . |
| Outcome: | The proposed method improves acceptance rates with only linear computational overhead. |
TokenTiming: A Dynamic Alignment Method for Universal Speculative Decoding Model Pairs (2026.acl-long)
Copied to clipboard
| Challenge: | Speculative decoding (SD) is a useful tool for accelerating large language models . but its utility is limited by a fundamental constraint: draft and target models must share the same vocabulary . |
| Approach: | They propose an algorithm that uses a draft token sequence to get a new target token sequence and then uses DTW to build a mapping to transfer probability distributions. |
| Outcome: | The proposed method shows 1.57x speedup on various tasks. |
Draft Model Knows When to Stop: Self-Verification Speculative Decoding for Long-Form Generation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Conventional speculative decoding methods use a predefined length policy for proposing drafts, but the reality deviates from this assumption. |
| Approach: | They propose a self-verification length policy that adaptively determines the lengths of draft sequences by referring to the draft entropy. |
| Outcome: | The proposed method achieves 17% speedup on MT-Bench and 22% speedup in long-form reasoning. |
RACER: Retrieval-Augmented Contextual Rapid Speculative Decoding (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for decoding large language models generate one token per step, causing high inference latency. |
| Approach: | They propose a method that integrates retrieved exact patterns with logit-driven future cues. |
| Outcome: | Experiments on Spec-Bench, HumanEval, and MGSM-ZH show that RACER outperforms training-free methods and accelerates inference. |
SAM Decoding: Speculative Decoding via Suffix Automaton (2025.acl-long)
Copied to clipboard
| Challenge: | Speculative decoding (SD) methods are inefficient and rely on single retrieval resources. |
| Approach: | They propose a retrieval-based speculative decoding method that adapts the suffix automaton for efficient draft generation by utilizing the generating text sequence and static text corpus. |
| Outcome: | The proposed method can find the longest suffix match and can be integrated with existing methods to generalize to broader domains. |
Faster Speculative Decoding via Effective Draft Decoder with Pruned Candidate Tree (2025.acl-long)
Copied to clipboard
| Challenge: | Effective Draft Decoder (EDD) is a powerful decoding method that generates more accurate draft tokens by leveraging the encoding results as soft prompts. |
| Approach: | They propose an effective draft decoder which treats the LLM as a powerful encoder and generates more accurate draft tokens by leveraging the encoding results as soft prompts. |
| Outcome: | The proposed method significantly improves the performance of large language models and reduces inference latency. |
EMS-SD: Efficient Multi-sample Speculative Decoding for Accelerating Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Speculative decoding is a key technique for enhancing the inference speed of Large Language Models. |
| Approach: | They propose a method that adds padding tokens to ensure that the number of new tokens remains consistent across samples. |
| Outcome: | The proposed method can handle the issue of inconsistent prediction tokens without adding padding tokens. |
Draft
& Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for accelerating Large Language Models have been criticized for their inference costs and inefficient decoding. |
| Approach: | They propose a self-speculative decoding approach for accelerating Large Language Models without an auxiliary model. |
| Outcome: | The proposed method achieves a speedup of up to 1.99 with no additional neural network training and no extra memory footprint. |