PToco: Prefix-based Token-level Collaboration Enhances Reasoning for Multi-LLMs (2025.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to collaboration between multiple Large Language Models (LLMs) rely on highly capable models with strong self-reflection abilities or are limited to models sharing the same tokenizer. |
| Approach: | They propose a mechanism that enables collaboration among less capable LLMs independent of tokenizer differences. |
| Outcome: | The proposed mechanism improves performance over individual models and generalizes well across different quantities and sizes of participating models. |
Similar Papers
Improving Consistency in LLM Inference using Probabilistic Tokenization (2025.findings-naacl)
Copied to clipboard
| Challenge: | Prior work has shown that probabilistic tokenizations can generate multiple tokenization of the same input string. |
| Approach: | They propose a method to leverage the multiple tokenization capabilities of modern LLM tokenizers. |
| Outcome: | The proposed method improves the self-consistency of large language models by generating multiple tokenizations. |
Harnessing Consistency for Robust Test-Time LLM Ensemble (2026.findings-eacl)
Copied to clipboard
Zhichen Zeng, Qi Yu, Xiao Lin, Ruizhong Qiu, Xuying Ning, Tianxin Wei, Yuchen Yan, Jingrui He, Hanghang Tong
| Challenge: | Existing efforts to improve LLM ensemble quality have focused on model consistency, but failures are often due to heterogeneous tokenization schemes and varying model expertise. |
| Approach: | They propose a plug-and-play technique that harnesses model consistency for robust LLM ensemble. |
| Outcome: | The proposed technique improves ensemble performance and robustness against erroneous signals. |
TARo: Token-level Adaptive Routing for LLM Test-time Alignment (2026.findings-acl)
Copied to clipboard
Arushi Rai, Qiang Zhang, Hanqing Zeng, Yunkai Zhang, Dipesh Tamboli, Xiangjun Fan, Zhuokai Zhao, Lizhu Zhang
| Challenge: | Large language models (LLMs) exhibit strong reasoning capabilities but typically require expensive post-training to reach high performance. |
| Approach: | They propose to use token-level Adaptive Routing to steer frozen LLMs toward structured reasoning entirely at inference time. |
| Outcome: | Extensive experiments show that TARo significantly improves reasoning performance by up to +22.4% over base model and +8.4% . |
Dynamic Collaboration of Multi-Language Models based on Minimal Complete Semantic Units (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to enhance reasoning capabilities of language models are expensive and often lack the ability to perform complex reasoning tasks. |
| Approach: | They propose a token-level multi-model collaboration strategy to enhance reasoning capabilities in language models by selecting the optimal tokens from the next token distributions. |
| Outcome: | The proposed method is superior to existing methods and will be released soon. |
Hit the Sweet Spot! Span-Level Ensemble for Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | a recent study focused on sample-level and token-level ensembles, which hinder dynamic correction and enhancement of outputs during the generation process. |
| Approach: | They propose a span-level ensemble method that balances real-time adjustments and accurate ensemble decisions. |
| Outcome: | The proposed method improves performance across language generation tasks significantly. |
Token-Level Policy Optimization: Linking Group-Level Rewards to Token-Level Aggregation via sequence-level likelihood (2026.acl-long)
Copied to clipboard
| Challenge: | Group Relative Policy Optimization (GRPO) has significantly advanced the reasoning ability of large language models (LLMs). |
| Approach: | They propose a token-level framework that leverages sequence-level likelihood to link group-level rewards with individual tokens via token- level aggregation and introduces a KL-Divergence mask constraint that targets tokens with positive advantages and decreasing entropy to mitigate abrupt policy updates. |
| Outcome: | Experiments show that TEPO achieves state-of-the-art performance on mathematical reasoning benchmarks and reduces convergence time by 50% compared with GRPO/DAPO. |
Learn Your Tokens: Word-Pooled Tokenization for Language Modeling (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Language models typically tokenize text into subwords, using a deterministic, hand-engineered heuristic of combining characters into longer surface-level strings such as ‘ing’ or whole words. |
| Approach: | They propose a 'learn your tokens' scheme which pooles bytes/characters into word representations and decodes individual characters/bytes per word in parallel. |
| Outcome: | The proposed tokenizer outperforms subword models and byte/character models over the word boundary and outperformed on rare words by a factor of 30! |
The Strawberry Problem: Emergence of Character-level Understanding in Tokenized Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models fail at simple character-level tasks due to low mutual information, study finds . authors propose a lightweight architectural modification that improves character- level reasoning . |
| Approach: | They propose a lightweight architectural modification that improves character-level reasoning while preserving the inductive advantages of subword models. |
| Outcome: | The proposed model improves character-level reasoning while preserving the advantages of subword models. |
Explain-Analyze-Generate: A Sequential Multi-Agent Collaboration Method for Complex Reasoning (2025.coling-main)
Copied to clipboard
| Challenge: | Multiagent debate (MAD) is a popular approach for large language models . however, the performance of LLMs is suboptimal in complex reasoning scenarios . |
| Approach: | They propose a sequential collaboration framework to enable agents to provide constructive assistance to peers by decomposing complex tasks into essential subtasks. |
| Outcome: | The proposed framework achieves the highest performance on 19 out of 23 tasks and lower costs compared to MAD. |
Token Knowledge: A New Perspective For Knowledge in Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Predicting the presence and absence of certain knowledge in large language models could aid hallucination avoidance. |
| Approach: | They propose a token knowledge dataset construction method and use the intermediate states during inference to train probes. |
| Outcome: | The proposed method increases the model's latent potential by 60% to 90% with strong out-of-distribution generalization by training on just a few dozen prompts. |