GUARD: Glocal Uncertainty-Aware Robust Decoding for Effective and Efficient Open-Ended Text Generation (2025.findings-emnlp)
Copied to clipboard
Yuanhao Ding, Esteban Garces Arias, Meimingwei Li, Julian Rodemann, Matthias Aßenmacher, Danlu Chen, Gaojuan Fan, Christian Heumann, Chongsheng Zhang
| Challenge: | GUARD is a self-adaptive decoding method that balances coherence with diversity in open-ended text generation. |
| Approach: | They propose a self-adaptive decoding method that balances coherence and diversity . they combine global entropy estimates with local entropic deviations to integrate uncertainty . |
| Outcome: | GUARD achieves a good balance between diversity and coherence while exhibiting significant improvements in generation speed. |
Similar Papers
CoCoA: Confidence- and Context-Aware Adaptive Decoding for Resolving Knowledge Conflicts in Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing contrastive decoding methods that handle conflict lack adaptability and can degrade performance in low conflict settings. |
| Approach: | They propose a token-level algorithm for principled conflict resolution and enhanced faithfulness that resolves conflict by utilizing confidence-aware measures and the generalized divergence between parametric and contextual distributions. |
| Outcome: | The proposed algorithm achieves 9.2 points on average in QA, summarization, and long-form question answering (LFQA) benchmarks and improves factuality by 2.5 points on the key benchmarks. |
Uncertainty-Aware Contrastive Decoding (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models excel in a wide range of tasks, but generating factually accurate outputs remains a challenge. |
| Approach: | They propose a method that dynamically adjusts model contributions at each decoding step based on uncertainty. |
| Outcome: | The proposed method significantly improves factual accuracy and reliability over existing methods. |
Balancing Diversity and Risk in LLM Sampling: How to Select Your Method and Parameter for Open-Ended Text Generation (2025.acl-long)
Copied to clipboard
| Challenge: | Recent studies suggest that sampling-based decoding strategies can be used to optimize the output of Large Language Models (LLMs) . previous studies have shown that likelihood-maximization produces degenerate text which contains repetitive loops and incoherent context, especially in open-ended tasks. |
| Approach: | They propose to use a prefix tree to estimate the intrinsic capacity of a truncation sampling method by considering the trade-off between diversity and risk at each decoding step. |
| Outcome: | The proposed method is based on a prefix tree which preserves the context of a full sentence. |
Adaptive Contrastive Search: Uncertainty-Guided Decoding for Open-Ended Text Generation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to decode text to the most probable sequence have been proposed to address these challenges by improving coherence, diversity, and resemblance to human-generated text. |
| Approach: | They propose a novel decoding strategy that extends contrastive search by incorporating an adaptive degeneration penalty informed by the model’s estimated uncertainty at each generation step. |
| Outcome: | The proposed approach improves creativity and coherence while maintaining coherency across model architectures, languages, and datasets. |
SecDecoding: Steerable Decoding for Safer LLM Generation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing decoding-time defense methods suffer from limited generalization, high computational overhead, or significant utility degradation. |
| Approach: | They propose a decoding-time defense framework that leverages a pair of small contrastive models to estimate token-level safety signals by measuring divergence in their output distributions. |
| Outcome: | The proposed framework achieves near-zero attack success rates against a wide spectrum of advanced jailbreak attacks while maintaining the model’s helpfulness with minimal degradation. |
Invisible Entropy: Towards Safe and Efficient Low-Entropy LLM Watermarking (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to watermark low-entropy content are expensive and risky . IE reduces parameter size by 99% while achieving performance on par with state-of-the-art methods . |
| Approach: | They propose a logit-based watermarking paradigm that uses entropy-based features to predict whether the next token is high or low. |
| Outcome: | The proposed method reduces parameter size by 99% while achieving performance on par with state-of-the-art methods. |
Why Not Act on What You Know? Unleashing Safety Potential of LLMs via Self-Aware Guard Enhancement (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated impressive capabilities across various tasks but are vulnerable to meticulously crafted jailbreak attacks. |
| Approach: | They propose a training-free defense strategy to align LLMs’ strong safety discrimination performance with their relatively weaker safety generation ability. |
| Outcome: | The proposed strategy achieves an average 99% success rate against numerous complex and covert jailbreak methods while maintaining helpfulness on general benchmarks. |
SpecBound: Adaptive Bounded Self-Speculation with Layer-wise Confidence Calibration (2026.findings-acl)
Copied to clipboard
| Challenge: | Speculative decoding has emerged as a promising approach to accelerate autoregressive inference in large language models. |
| Approach: | They propose a self-draft framework that suppresses spurious confidence via layer-wise temperature annealing in early-exit decision and adaptively bounds speculation length based on token-wise decoding difficulty. |
| Outcome: | The proposed framework suppresses spurious confidence and bounds speculation length based on token-wise decoding difficulty. |
Odysseus Navigates the Sirens’ Song: Dynamic Focus Decoding for Factual and Diverse Open-Ended Text Generation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing decoding methods struggle to balance factuality and diversity . Deterministic decoding approaches suffer from degeneration and lack of diversity - a problem that is not addressed by the current literature. |
| Approach: | They propose a plug-and-play stochastic approach that adjusts decoding focus based on distributional differences across layers, leveraging the modular nature of factual knowledge within LLMs. |
| Outcome: | Extensive experiments on seven datasets show that DFD significantly improves performance. |
Speculative Decoding via Early-exiting for Faster LLM Inference with Thompson Sampling Control Mechanism (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to generate draft tokens in large language models are expensive and resource-intensive. |
| Approach: | They propose an approach to generate draft tokens using a segment of the LLM and a self-distillation method to enhance the quality of draft token. |
| Outcome: | The proposed approach generates draft tokens using a segment of the LLM and a self-distillation method to improve quality and speed up generation. |