| Challenge: | Generating diverse sequences exhibit semantically one-to-many relationships between source and target sequences. |
| Approach: | They propose to separate diversification from generation using a general plug-and-play module that wraps around and guides an existing encoder-decoder model. |
| Outcome: | The proposed method shows that diversification and generation are separate steps in the same model and that the model is robust. |
Similar Papers
A Well-Composed Text is Half Done! Composition Sampling for Diverse Conditional Generation (2022.acl-long)
Copied to clipboard
Shashi Narayan, Gonçalo Simões, Yao Zhao, Joshua Maynez, Dipanjan Das, Michael Collins, Mirella Lapata
| Challenge: | Composition Sampling is a simple but effective method to generate diverse outputs for conditional generation of higher quality compared to previous stochastic decoding strategies. |
| Approach: | They propose a method to generate diverse outputs for conditional generation . they use a plan-based neural generation model that is trained to create a composition of the output and then generate by conditioning on it and the input. |
| Outcome: | The proposed method avoids text degeneration by first sampling a composition in the form of an entity chain and then using beam search to generate the best possible text grounded to this entity chain. |
Mixup Decoding for Diverse Machine Translation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for generating multiple translations for source and target languages neglect the one-to-many mapping between the source and the target languages. |
| Approach: | They propose a method to generate different translations for the input sentence by linearly interpolating it with different sentence pairs sampled from the training corpus during decoding. |
| Outcome: | Experiments on WMT’16 en-ro, WMT'14 en de, and WMT ‘17 zh-en show that the proposed method outperforms all previous diverse machine translation methods. |
Principled Content Selection to Generate Diverse and Personalized Multi-Document Summaries (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models exhibit the _”lost in the middle” phenomenon when they are unevenly attending to different parts of the provided context. |
| Approach: | They propose principled content selection as a way to increase source coverage . they use determinantal point processes to prioritize diverse content . |
| Outcome: | The proposed method improves source coverage on the DiverseSumm benchmark. |
Fine-grained Contrastive Learning for Definition Generation (2022.aacl-main)
Copied to clipboard
| Challenge: | Recent pre-trained transformer-based definition generation models lack effective representation learning to contain full semantic components of the given word, leading to under-specific definitions. |
| Approach: | They propose a novel contrastive learning method that encourages the model to capture more detailed semantic representations from the definition sequence encoding. |
| Outcome: | The proposed method could generate more specific definitions compared with state-of-the-art models. |
Twist Decoding: Diverse Generators Guide Each Other (2022.emnlp-main)
Copied to clipboard
Jungo Kasai, Keisuke Sakaguchi, Ronan Le Bras, Hao Peng, Ximing Lu, Dragomir Radev, Yejin Choi, Noah A. Smith
| Challenge: | Using a variety of language generation models, ensembling models is challenging during inference. |
| Approach: | They propose a method that decodes text models that do not assume a shared vocabulary, tokenization or generation order. |
| Outcome: | The proposed method outperforms models decoded in isolation over various scenarios. |
Comparison of Diverse Decoding Methods from Conditional Language Models (P19-1)
Copied to clipboard
| Challenge: | Conditional language models can generate a diverse set of outputs, but for open-ended tasks, beam search is ill-suited to generating a set of diverse sequences. |
| Approach: | They propose a method where we over-sample candidates and use clustering to remove similar sequences to achieve high diversity without sacrificing quality. |
| Outcome: | The proposed method over-samples candidates and removes similar sequences to achieve high diversity without sacrificing quality. |
Structurally Diverse Sampling for Sample-Efficient Training and Comprehensive Evaluation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to generalize compositionally are inadequate, but there is no evidence for this. |
| Approach: | They propose a model-agnostic algorithm for subsampling instances with diverse structures from a labeled instance pool with structured outputs. |
| Outcome: | The proposed algorithm leads to comparable or better generalization than prior algorithms in 9 out of 10 dataset-split type pairs. |
KCS: Diversify Multi-hop Question Generation with Knowledge Composition Sampling (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to multi-hop question answering focus on generating simple questions and neglecting the integration of essential knowledge, such as relevant sentences within documents. |
| Approach: | They propose a framework to expand the diversity of generated multi-hop questions by sampling varied knowledge compositions within a given context. |
| Outcome: | The proposed framework improves the overall accuracy of knowledge composition selection by 3.9% on hotpotQA and 2WikiMultihopQA datasets. |
MoR: Better Handling Diverse Queries with a Mixture of Sparse, Dense, and Human Retrievers (2025.emnlp-main)
Copied to clipboard
| Challenge: | Different retrievers offer distinct, often complementary signals, but they are not optimal for all queries. |
| Approach: | They propose a zero-shot, weighted combination of heterogeneous retrievers . they validate this intuition by incorporating specialized non-oracle human information sources . |
| Outcome: | Experiments show that a mixture of heterogeneous retrievers outperforms each retriever and larger models by +10.8% and +3.9% on average. |
Select and Attend: Towards Controllable Content Selection in Text Generation (D19-1)
Copied to clipboard
| Challenge: | Recent neural network models conflate content selection and surface realization into a black-box architecture, resulting in content to be described in text cannot be explicitly controlled. |
| Approach: | They propose to decouple content selection from the decoder to allow finer-grained control over the generation. |
| Outcome: | The proposed model can be trained end-to-end without human annotations and achieves promising results in data-totext and headline generation tasks. |