How Decoding Strategies Affect the Verifiability of Generated Text (2020.findings-emnlp)
Copied to clipboard
Luca Massarelli, Fabio Petroni, Aleksandra Piktus, Myle Ott, Tim Rocktäschel, Vassilis Plachouras, Fabrizio Silvestri, Sebastian Riedel
| Challenge: | Recent advances in pre-trained language models have generated text of an increasingly high quality. |
| Approach: | They propose a decoding strategy that produces less repetitive and more verifiable text. |
| Outcome: | The proposed method produces less repetitive and more verifiable text than previously used decoding strategies. |
Similar Papers
On Decoding Strategies for Neural Text Generators (2022.tacl-1)
Copied to clipboard
| Challenge: | a recent study suggests that decoding strategies may be more important than the model architecture itself when generating text from probabilistic models. |
| Approach: | They propose to measure changes in attributes of generated text as a function of decoding strategy and task using human and automatic evaluation. |
| Outcome: | The proposed study shows that decoding strategies do not always transfer across tasks . authors show that the differences in attributes are not always consistent across tasks, they say . |
Decoding Uncertainty: The Impact of Decoding Strategies for Uncertainty Estimation in Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Decoding strategies affect the probability distribution underlying the output of a language model and can therefore affect both generation quality and uncertainty. |
| Approach: | They investigate the impact of decoding strategies on uncertainty estimation in large language models . |
| Outcome: | The proposed methods improve the uncertainty estimation of large language models by reducing repetition. |
Decoding Decoded: Understanding Hyperparameter Effects in Open-Ended Text Generation (2025.coling-main)
Copied to clipboard
| Challenge: | Generative large language models generate a high-dimensional probability distribution over all tokens in their vocabulary. |
| Approach: | They conduct extensive sensitivity analyses to determine how hyperparameter choices shape the outputs of generative large language models. |
| Outcome: | The proposed methods influence the distribution of diversity and coherence metrics in human-written text, but the optimal configurations vary across models and tasks. |
Exploring the Limitations of Detecting Machine-Generated Text (2025.coling-main)
Copied to clipboard
| Challenge: | Recent advances in the quality of the generation of text by large language models have spurred research into identifying machine-generated text. |
| Approach: | They audit classification performance for detecting machine-generated text by evaluating on texts with varying writing styles. |
| Outcome: | The proposed methods are highly sensitive to stylistic changes and complexity, and in some cases degrade entirely to random classifiers. |
Local Normalization Distortion and the Thermodynamic Formalism of Decoding Strategies for Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing decoding strategies for language models are based on heuristics and are difficult to improve. |
| Approach: | They propose a theory of decoding strategies for language models by expressing popular decoding algorithms as equilibrium states and stating objective functions they optimize. |
| Outcome: | The proposed method is based on ergodic theory and is difficult to improve . it shows that local normalization distortion is a fundamental defect of decoding strategies . |
Automatic Detection of Generated Text is Easiest when Humans are Fooled (2020.acl-main)
Copied to clipboard
| Challenge: | Recent advances in neural language modelling make it possible to rapidly generate vast amounts of human-sounding text. |
| Approach: | They compare decoding methods with popular sampling-based decoding strategies . they show that multi-sentence excerpts can fool expert human raters over 30% of the time . |
| Outcome: | The proposed methods improve with longer excerpt length, but multi-sentence excerpts fool human raters over 30% of the time. |
GenProve: Learning to Generate Text with Fine-Grained Provenance (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for large language models (LLMs) are coarse-grained and fail to distinguish between direct quotes and complex reasoning. |
| Approach: | They propose a framework that combines supervised fine-tuning and group relative policy optimization to generate fluent answers while simultaneously producing sentence-level provenance triples. |
| Outcome: | The proposed framework outperforms 14 strong large language models in joint evaluation. |
Verifiable Generation with Subsentence-Level Fine-Grained Citations (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing work mainly targets the generation of sentence-level citations, lacking specificity about which parts of a sentence are backed by the cited sources. |
| Approach: | They propose to use subsentence-level fine-grained citations to generate more precise location of generated content supported by the cited sources. |
| Outcome: | The proposed model improves the accuracy and trustworthiness of large language models by allowing users to trace the information back to its source and verify its correctness. |
Evaluating Verifiability in Generative Search Engines (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing generative search engines are rapidly gaining users, according to a new study . existing systems are poorly cited and lack reliability, a study finds . |
| Approach: | They conduct human evaluations of four popular generative search engines . they find that existing generative engines are fluent and appear informative . |
| Outcome: | The results show that existing generative search engines are not reliable and often contain unsupported statements and inaccurate citations. |
How Sampling Affects the Detectability of Machine-written texts: A Comprehensive Study (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Recent detectors report near-perfect accuracy, often boasting AUROC scores above 99%, but these claims typically assume fixed generation settings, leaving open the question of how robust such systems are to changes in decoding strategies. |
| Approach: | They examine how sampling-based decoding impacts detectability with a focus on how subtle variations in a model’s (sub)word-level distribution affect detection performance. |
| Outcome: | The proposed framework systematically examines how sampling-based decoding impacts detectability, with a focus on how subtle variations in a model’s (sub)word-level distribution affect detection performance. |