Diversity Explains Inference Scaling Laws: Through a Case Study of Minimum Bayes Risk Decoding (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies show that MBR decoding improves model generation performance . however, the theoretical underpinnings of these results remain uncertain . |
| Approach: | They propose a theoretical interpretation of MBR decoding from the perspective of bias–diversity decomposition. |
| Outcome: | The proposed method improves the quality estimation of hypotheses by decomposing bias and diversity into two main factors. |
Similar Papers
Theoretical Guarantees for Minimum Bayes Risk Decoding (2025.acl-long)
Copied to clipboard
| Challenge: | Minimum Bayes Risk (MBR) decoding is a decision rule used to generate sequences from autoregressive probability models (e.g., LLMs). |
| Approach: | They propose to use minimum bayes risk (MBR) decoding to optimize output selection by maximizing expected utility value of an underlying human distribution. |
| Outcome: | The proposed method is effective even though the language space Y is larger than the hypothesis set. |
Generating Diverse and High-Quality Texts by Minimum Bayes Risk Decoding (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing decoding algorithms to generate diverse outputs are based on beam search or random sampling, thus their output quality is capped by these underlying decoding methods. |
| Approach: | They propose to add a diversity penalty to MBR decoding and a clustering problem to create diversity-promoting decoding algorithms by enforcing diversity objectives. |
| Outcome: | The proposed method achieves a better trade-off than the diverse beam search and sampling algorithms overall. |
The Impact of Inference Acceleration on Bias of LLMs (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent work suggests strategies to increase inference efficiency with LLMs . however, these strategies may inadvertently lead to some side-effects. |
| Approach: | They propose to optimize inference acceleration strategies such as quantization, pruning, and caching to reduce inference cost and latency while maintaining predictive performance. |
| Outcome: | The proposed strategies reduce cost and latency while maintaining predictive performance while preserving the model size. |
Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) currently exhibits biases such as producing translations that are too short and overgenerating frequent words. |
| Approach: | They propose to use minimum bayes risk decoding instead of beam search to investigate the effects of beam decoding on unbiased samples. |
| Outcome: | The proposed method improves on a number of previously reported biases and failure cases of beam search on unbiased samples. |
Faster Minimum Bayes Risk Decoding with Confidence-based Pruning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Minimum Bayes risk (MBR) decoding is a decision rule for conditional sequence generation tasks. |
| Approach: | They propose an algorithm which grows the number of samples used to estimate utility . it prunes hypotheses that are unlikely to have the highest utility based on bootstrap sampling . |
| Outcome: | The proposed method outperforms beam search in conditional language generation and neural machine translation tasks while being statistically indistinguishable from other proposed methods. |
On the True Distribution Approximation of Minimum Bayes-Risk Decoding (2024.naacl-short)
Copied to clipboard
| Challenge: | Minimum Bayes-risk (MBR) decoding has recently gained renewed attention in text generation. |
| Approach: | They propose to use anomaly detection to measure the degree of approximation by sampling texts from a model and selecting the text with the highest similarity to the others. |
| Outcome: | The proposed method shows that previous hypotheses about samples do not correlate well with the variation, but the results support the core assumption of MBR decoding. |
Structure-Conditional Minimum Bayes Risk Decoding (2025.emnlp-main)
Copied to clipboard
| Challenge: | Minimum Bayes Risk (MBR) decoding has been used in machine translation for many years. |
| Approach: | They propose three adaptations to the minimum bayes risk utility function to make it more sensitive to structural variability in the outcome space. |
| Outcome: | The proposed adaptations significantly improve generation quality by up to 13.7 percentage points in win rate. |
Epsilon Sampling Rocks: Investigating Sampling Strategies for Minimum Bayes Risk Decoding for Machine Translation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in machine translation (MT) have shown that minimum bayes risk decoding can be a powerful alternative to beam search. |
| Approach: | They propose to use epsilon-sampling to prune away all tokens with a smaller probability mass. |
| Outcome: | The proposed method outperforms beam search decoding and other methods in four languages. |
Follow the Wisdom of the Crowd: Effective Text Generation via Minimum Bayes Risk Decoding (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing text decoding methods struggle to produce high-quality text . Greedy and beam search suffer from text degeneration and linguistic diversity issues . |
| Approach: | They propose a family of decoding methods based on minimum bayes risk minimization to address diversity-quality trade-offs in open-ended natural-language generation. |
| Outcome: | The proposed methods improve diversity-quality trade-offs on open-ended natural-language generation tasks. |
mbrs: A Library for Minimum Bayes Risk Decoding (2024.emnlp-demo)
Copied to clipboard
| Challenge: | Minimum Bayes risk (MBRS) decoding is a decision rule of text generation tasks that outperforms conventional maximum a posteriori (MAP) decoders by selecting high-quality outputs based on quality or preference rather than probability. |
| Approach: | They propose to use minimum bayes risk (MBRS) decoding to determine outputs based on quality rather than probability. |
| Outcome: | MBRS is an MIT-licensed open-source project with a focus on speed, reproducibility, and extensibility. |