| Challenge: | Large language models (LLMs) enable an extraordinary range of applications, yet their adoption is limited by a fundamental concern: users must send sensitive data to remote model providers. |
| Approach: | They propose an efficient homomorphic encryption algorithm for inverse transform sampling that allows private token sampling under HE. |
| Outcome: | The proposed method samples tokens in 0.14 seconds on GPU, achieving a 100 latency improvement over prior work. |
Similar Papers
Balancing Diversity and Risk in LLM Sampling: How to Select Your Method and Parameter for Open-Ended Text Generation (2025.acl-long)
Copied to clipboard
| Challenge: | Recent studies suggest that sampling-based decoding strategies can be used to optimize the output of Large Language Models (LLMs) . previous studies have shown that likelihood-maximization produces degenerate text which contains repetitive loops and incoherent context, especially in open-ended tasks. |
| Approach: | They propose to use a prefix tree to estimate the intrinsic capacity of a truncation sampling method by considering the trade-off between diversity and risk at each decoding step. |
| Outcome: | The proposed method is based on a prefix tree which preserves the context of a full sentence. |
Watermarking Large Language Models: An Unbiased and Low-risk Method (2025.acl-long)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have highlighted the risk of misusing them, raising the need for accurate detection of LLM-generated content. |
| Approach: | They propose a method to inject imperceptible identifiers into large language models (LLMs) this method is unbiased and preserves the original token distribution in expectation . |
| Outcome: | The proposed method preserves the original token distribution in expectation and has lower risk of producing unsatisfactory outputs in low-entropy scenarios compared to existing unbiased watermarks. |
Private prediction for large-scale synthetic text generation (2024.findings-emnlp)
Copied to clipboard
Kareem Amin, Alex Bie, Weiwei Kong, Alexey Kurakin, Natalia Ponomareva, Umar Syed, Andreas Terzis, Sergei Vassilvitskii
| Challenge: | Existing approaches to generate differentially private text using large language models are classified into several categories. |
| Approach: | They propose a private prediction framework that generates differentially private synthetic text using large language models via private prediction. |
| Outcome: | The proposed approach generates high-quality synthetic data points at reasonable privacy levels while protecting the privacy of users who contributed to the dataset. |
Speculative Decoding via Early-exiting for Faster LLM Inference with Thompson Sampling Control Mechanism (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to generate draft tokens in large language models are expensive and resource-intensive. |
| Approach: | They propose an approach to generate draft tokens using a segment of the LLM and a self-distillation method to enhance the quality of draft token. |
| Outcome: | The proposed approach generates draft tokens using a segment of the LLM and a self-distillation method to improve quality and speed up generation. |
Sparse-to-Dense: A Free Lunch for Lossless Acceleration of Video Understanding in LLMs (2025.acl-short)
Copied to clipboard
| Challenge: | Recent advances in Video Large Language Models (Video-LLMs) have achieved exceptional performance on tasks like video question answering and captioning. |
| Approach: | They propose a decoding strategy that leverages sparse top-K attention and dense full attention to accelerate Video-LLMs without loss. |
| Outcome: | The proposed approach achieves a 1.94 walltime speedup in video processing. |
Turning Trash into Treasure: Accelerating Inference of Large Language Models with Token Recycling (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) generate only one token at each decoding step, leading to high latency. |
| Approach: | They propose a speculative decoding paradigm that stores tokens in an adjacency matrix and employs a breadth-first-search algorithm to construct a draft tree. |
| Outcome: | The proposed method outperforms existing train-free methods by 30% and even a training method by 25%. |
Powerformer: Efficient and High-Accuracy Privacy-Preserving Language Model with Homomorphic Encryption (2025.acl-long)
Copied to clipboard
| Challenge: | a new privacy-preserving language model, Powerformer, is designed to reduce computation overhead while maintaining model performance. |
| Approach: | They propose an efficient homomorphic encryption-based privacy-preserving language model . it incorporates three key techniques to optimize encrypted computations . |
| Outcome: | The proposed model achieves 45% reduction in computation time compared to state-of-the-art models . authors say the model preserves data privacy and AI capabilities in MLaaS environments . |
Evolving Sparsity: Leveraging Token Importance Dynamics for Efficient LLM Decoding with Sparse Attention (2026.acl-long)
Copied to clipboard
| Challenge: | Efficient long-context inference remains a major challenge for large language models (LLMs), as the cost of attention computation during auto-regressive decoding grows linearly with the context length. |
| Approach: | They propose to model token importance as a dynamic process that evolves over decoding steps and propagates through model layers. |
| Outcome: | The proposed method outperforms baseline sparse attention methods and achieves speedups of up to 5.36 for attention latency and 2.33 for end-to-end decoding. |
EMS-SD: Efficient Multi-sample Speculative Decoding for Accelerating Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Speculative decoding is a key technique for enhancing the inference speed of Large Language Models. |
| Approach: | They propose a method that adds padding tokens to ensure that the number of new tokens remains consistent across samples. |
| Outcome: | The proposed method can handle the issue of inconsistent prediction tokens without adding padding tokens. |
Exploring the Hidden Capacity of LLMs for One-Step Text Generation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) can reconstruct surprisingly long texts via autoregressive generation from just one trained input embedding. |
| Approach: | They show that large language models can reconstruct surprisingly long texts via autoregressive generation from just one trained input embedding. |
| Outcome: | The proposed model can generate hundreds of accurate tokens in one token-parallel forward pass, when provided with only two learned embeddings. |