Local Normalization Distortion and the Thermodynamic Formalism of Decoding Strategies for Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing decoding strategies for language models are based on heuristics and are difficult to improve. |
| Approach: | They propose a theory of decoding strategies for language models by expressing popular decoding algorithms as equilibrium states and stating objective functions they optimize. |
| Outcome: | The proposed method is based on ergodic theory and is difficult to improve . it shows that local normalization distortion is a fundamental defect of decoding strategies . |
Similar Papers
Local and Global Decoding in Text Generation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Text generation relies heavily on decoding algorithms that sample strings from a language model distribution. |
| Approach: | They propose to introduce globally-normalised versions of traditional decoding methods and propose an independent Metropolis-Hastings algorithm to approximate sampling from globally-averaged distributions without explicitly computing them. |
| Outcome: | The proposed method approximates the distributions without explicitly computing them. |
Decoding Decoded: Understanding Hyperparameter Effects in Open-Ended Text Generation (2025.coling-main)
Copied to clipboard
| Challenge: | Generative large language models generate a high-dimensional probability distribution over all tokens in their vocabulary. |
| Approach: | They conduct extensive sensitivity analyses to determine how hyperparameter choices shape the outputs of generative large language models. |
| Outcome: | The proposed methods influence the distribution of diversity and coherence metrics in human-written text, but the optimal configurations vary across models and tasks. |
Decoding Uncertainty: The Impact of Decoding Strategies for Uncertainty Estimation in Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Decoding strategies affect the probability distribution underlying the output of a language model and can therefore affect both generation quality and uncertainty. |
| Approach: | They investigate the impact of decoding strategies on uncertainty estimation in large language models . |
| Outcome: | The proposed methods improve the uncertainty estimation of large language models by reducing repetition. |
How Decoding Strategies Affect the Verifiability of Generated Text (2020.findings-emnlp)
Copied to clipboard
Luca Massarelli, Fabio Petroni, Aleksandra Piktus, Myle Ott, Tim Rocktäschel, Vassilis Plachouras, Fabrizio Silvestri, Sebastian Riedel
| Challenge: | Recent advances in pre-trained language models have generated text of an increasingly high quality. |
| Approach: | They propose a decoding strategy that produces less repetitive and more verifiable text. |
| Outcome: | The proposed method produces less repetitive and more verifiable text than previously used decoding strategies. |
A Thorough Examination of Decoding Methods in the Era of LLMs (2024.emnlp-main)
Copied to clipboard
| Challenge: | Decoding methods are essential for converting language models from next-token predictors into practical task solvers. |
| Approach: | They propose to evaluate decoding methods in general-purpose large language models . they find that decoding method performance is notably task-dependent . |
| Outcome: | The proposed methods perform task-dependently and are influenced by alignment, model size, and quantization. |
Locally Typical Sampling (2023.tacl-1)
Copied to clipboard
| Challenge: | a discrepancy in probabilistic language generators has puzzled the language generation community for years . |
| Approach: | They propose a method for local typical sampling to enforce a conditional entropy criterion for probabilistic models. |
| Outcome: | The proposed method can provide new insights into why high-probability texts can be dull or repetitive. |
On Decoding Strategies for Neural Text Generators (2022.tacl-1)
Copied to clipboard
| Challenge: | a recent study suggests that decoding strategies may be more important than the model architecture itself when generating text from probabilistic models. |
| Approach: | They propose to measure changes in attributes of generated text as a function of decoding strategy and task using human and automatic evaluation. |
| Outcome: | The proposed study shows that decoding strategies do not always transfer across tasks . authors show that the differences in attributes are not always consistent across tasks, they say . |
On the probability–quality paradox in language generation (2022.acl-short)
Copied to clipboard
| Challenge: | a recent study suggests that mode-seeking decoding methods produce unnatural language . a higher probability text is perceived as more human-like, but this is not the case with stochastic decoding . |
| Approach: | They propose to explain why higher-probability text is perceived as more human-like . they argue that human-type language should contain an amount of information close to entropy . |
| Outcome: | The proposed model produces lower-probability text, which is perceived as more human-like . elucidation of this hypothesis supports the hypothesis . |
Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding (2024.findings-acl)
Copied to clipboard
Heming Xia, Zhe Yang, Qingxiu Dong, Peiyi Wang, Yongqi Li, Tao Ge, Tianyu Liu, Wenjie Li, Zhifang Sui
| Challenge: | Large Language Models (LLMs) have a high inference latency stemming from autoregressive decoding. |
| Approach: | They propose a novel decoding paradigm that drafts multiple tokens and verifies them in parallel . they aim to provide a catalyst for further research on Speculative Decoding . |
| Outcome: | The proposed method drafts multiple tokens and verifies them in parallel . it can be used to accelerate inference in large language models. |
How Sampling Affects the Detectability of Machine-written texts: A Comprehensive Study (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Recent detectors report near-perfect accuracy, often boasting AUROC scores above 99%, but these claims typically assume fixed generation settings, leaving open the question of how robust such systems are to changes in decoding strategies. |
| Approach: | They examine how sampling-based decoding impacts detectability with a focus on how subtle variations in a model’s (sub)word-level distribution affect detection performance. |
| Outcome: | The proposed framework systematically examines how sampling-based decoding impacts detectability, with a focus on how subtle variations in a model’s (sub)word-level distribution affect detection performance. |