TreeBoN: Enhancing Inference-Time Alignment with Speculative Tree-Search and Best-of-N Sampling

Challenge:	Best-of-N (BoN) sampling generates multiple responses and selects the best one, achieving improved performance but with a high computational cost.
Approach:	They propose a framework that integrates a speculative tree-search strategy into Best-of-N (BoN) Sampling.
Outcome:	The proposed framework outperforms Best-of-N (BoN) sampling but has high computational cost . tree-search strategy reduces computational overhead while maintaining high output quality .

Similar Papers

Overview

Challenge

Approach

Outcome

Reward-Guided Tree Search for Inference Time Alignment of Large Language Models (2025.naacl-long)

Copied to clipboard

Chia-Yu Hung, Navonil Majumder, Ambuj Mehrish, Soujanya Poria

Semantic Text Processing

Challenge:	Inference-time computation methods enhance performance of Large Language Models by leveraging additional computational resources.
Approach:	They propose an inference-time alignment method that leverages a reward model to achieve alignment through reward-guided tree search.
Outcome:	The proposed method outperforms other inference-time alignment methods on two benchmarks . it achieves comparable performance to preference-tuned models on both benchmarks, authors show .

Reward-Shifted Speculative Sampling Is An Efficient Test-Time Weak-to-Strong Aligner (2025.emnlp-main)

Copied to clipboard

Bolian Li, Yanran Wu, Xinyu Luo, Ruqi Zhang

Green & Sustainable NLP

Challenge:	Recent research has focused on test-time alignment, where additional compute is allocated during inference to enhance LLM safety and reasoning capabilities.
Approach:	They propose a reward-shifted speculative sampling algorithm that aligns a draft model with human preferences while the target model remains unchanged.
Outcome:	The proposed algorithm achieves superior gold reward scores at a significantly reduced inference cost in test-time weak-to-strong alignment experiments.

Alignment-Augmented Speculative Decoding with Alignment Sampling and Conditional Verification (2025.emnlp-main)

Copied to clipboard

Jikai Wang, Zhenxu Tian, Juntao Li, Qingrong Xia, Xinyu Duan, Zhefeng Wang, Baoxing Huai, Min Zhang

Challenge:	Existing methods to accelerate autoregressive generation of large language models require training costs.
Approach:	They propose a training-free alignment-augmented speculative decoding algorithm . it leverages the output distribution obtained in the prefilling phase to provide more aligned draft candidates .
Outcome:	The proposed method increases the average generation score by 3.3 points for the LLaMA3 model.

Chain-in-Tree: Back to Sequential Reasoning in LLM Tree Search (2026.findings-acl)

Copied to clipboard

Xinzhe Li

Reasoning

Challenge:	Large language models excel at tasks such as mathematical and commonsense reasoning, but their performance improves further when additional test-time compute is allocated.
Approach:	They propose a plug-in framework that decides when to branch during search instead of expanding at every step.
Outcome:	The proposed framework reduces token generation, model calls, and runtime by 75-85% on GSM8K and Math500, with negligible or no accuracy loss.

Speculative Decoding Speed-of-Light: Optimal Lower Bounds via Branching Random Walks (2026.eacl-long)

Copied to clipboard

Sergey Pankratov, Dan Alistarh

Challenge:	Speculative generation has emerged as a promising technique to accelerate inference in large language models (LLMs) however, the fundamental limits on the achievable speedup remain poorly understood.
Approach:	They propose to draw a parallel token generation process and branching random walks to achieve the first "tight" lower bounds on the runtime of any deterministic speculative generation algorithm.
Outcome:	The proposed method reduces inference latency without altering the output distribution.

Turning Trash into Treasure: Accelerating Inference of Large Language Models with Token Recycling (2025.acl-long)

Copied to clipboard

Xianzhen Luo, Yixuan Wang, Qingfu Zhu, Zhiming Zhang, Xuanyu Zhang, Qing Yang, Dongliang Xu

Semantic Text Processing

Challenge:	Large Language Models (LLMs) generate only one token at each decoding step, leading to high latency.
Approach:	They propose a speculative decoding paradigm that stores tokens in an adjacency matrix and employs a breadth-first-search algorithm to construct a draft tree.
Outcome:	The proposed method outperforms existing train-free methods by 30% and even a training method by 25%.

SEED: Accelerating Reasoning Tree Construction via Scheduled Speculative Decoding (2025.coling-main)

Copied to clipboard

Zhenglin Wang, Jialong Wu, Yilong Lai, Congzhi Zhang, Deyu Zhou

Responsible & Trustworthy NLP

Challenge:	Large Language Models (LLMs) have remarkable emergent abilities across various tasks, yet their performance on complex reasoning and planning tasks remains suboptimal.
Approach:	They propose a tree-search-based reasoning framework that encourages the exploration of intermediate steps and a round-scheduled strategy to manage draft model dispatching.
Outcome:	The proposed framework improves runtime speed and GPU memory management concurrently and handles multiple iterations for thought generation and state evaluation.

Speculative Decoding for Multi-Sample Inference (2025.findings-emnlp)

Copied to clipboard

Yiwei Li, Jiayi Shi, Shaoxiong Feng, Peiwen Yuan, Xinglin Wang, Yueqi Zhang, Ji Zhang, Chuyi Tan, Boyuan Pan, Yao Hu, Kan Li

Reasoning

Challenge:	Speculative decoding method exploits consensus of parallel reasoning paths to synthesize high-quality draft tokens without auxiliary models or external databases.
Approach:	They propose a speculative decoding method that exploits the consensus of parallel reasoning paths to synthesize high-quality draft tokens without auxiliary models or external databases.
Outcome:	The proposed method exploits the intrinsic consensus of parallel reasoning paths to synthesize high-quality draft tokens without auxiliary models or databases.

SpecHub: Provable Acceleration to Multi-Draft Speculative Decoding (2024.emnlp-main)

Copied to clipboard

Ryan Sun, Tianyi Zhou, Xun Chen, Lichao Sun

Language Models

Challenge:	Large Language Models (LLMs) have limited inference speed due to sequential token generation . Spechub is a novel, efficient sampling-verification method for MDSD that improves acceptance rates with only linear computational overhead.
Approach:	They propose a method that uses a smaller draft model to generate multiple token sequences . Spechub generates 0.05-0.27 and 0.02-0.16 more tokens per step than RRS and RRS without replacement .
Outcome:	The proposed method improves acceptance rates with only linear computational overhead.

Token-level Inference-Time Alignment for Vision-Language Models (2026.findings-acl)

Copied to clipboard

Kejia Chen, Junjun Zheng, Jiawen Zhang, Manxi Lin, Xiao Pan, Jiacong Hu, Jian Lou, Zunlei Feng, Mingli Song

Language Models

Challenge:	Vision-Language Models (VLMs) often prioritize linguistic fluency over visual fidelity . despite widespread adoption, VLMs often exhibit a critical failure mode: hallucination .
Approach:	They propose a framework for Token-level Inference-Time Alignment that steers the decoding process without updating the base model parameters.
Outcome:	The proposed framework improves performance on 13 benchmarks across architectures . it boosts LLaVA-1.5-7B by 8.6% on MMVet and achieves a 74.0 MMStar score .