| Challenge: | Existing methods for conditional natural language generation are limited in their ability to produce controlled output. |
| Approach: | They propose to augment neural generation models with discrete control states learned through a structured latent-variable approach. |
| Outcome: | The proposed approach improves over benchmarks while providing fine-grained control. |
Similar Papers
Narrative Text Generation with a Latent Discrete Plan (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work on story generation has focused on generating consistent stories via story outlines using keywords or key phrases. |
| Approach: | They propose a deep latent variable model that first samples a sequence of anchor words, one per sentence in the story, as part of its generative process. |
| Outcome: | The proposed model gets favorable scores when evaluated on perplexity, diversity, and control of story via discrete plan. |
Generative Text Modeling through Short Run Inference (2021.eacl-main)
Copied to clipboard
| Challenge: | Latent variable models for text capture global semantic and syntactic features when trained correctly. |
| Approach: | They propose a short run dynamics for inference that initializes from the prior distribution of the latent variable and runs a small number of Langevin dynamics steps guided by its posterior distribution. |
| Outcome: | The proposed model is able to generate coherent sentences with smooth transition and shows no sign of posterior collapse. |
Using Structured Content Plans for Fine-grained Syntactic Control in Pretrained Language Model Generation (2022.coling-1)
Copied to clipboard
| Challenge: | Large pretrained language models can generate powerful text but cannot be controlled at a sub-sentential level. |
| Approach: | They propose to make such fine-grained control possible in pretrained LMs by generating text directly from a semantic representation, Abstract Meaning Representation (BART), which is augmented at the node level with syntactic control tags. |
| Outcome: | The proposed method can generate text from a semantic representation, which is augmented at the node level with syntactic control tags. |
Data-to-text Generation with Variational Sequential Planning (2022.tacl-1)
Copied to clipboard
| Challenge: | Recent advances in data-to-text generation have greatly facilitated the task of generating textual output from non-linguistic input. |
| Approach: | They propose a neural model enhanced with a planning component responsible for organizing high-level information in a coherent and meaningful way. |
| Outcome: | The proposed model outperforms baseline models and is sample-efficient in the face of limited training data. |
NeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic Constraints (2021.naacl-main)
Copied to clipboard
| Challenge: | Conditional text generation often requires lexical constraints, i.e., which words should or shouldn't be included in the output text. |
| Approach: | They propose an algorithm that enables neural language models to generate fluent text while satisfying complex lexical constraints. |
| Outcome: | The proposed algorithm outperforms existing methods on four benchmarks and shows that it handles any set of lexical constraints expressible under predicate logic while its asymptotic runtime is equivalent to conventional beam search. |
Facts2Story: Controlling Text Generation by Key Facts (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for story generation struggle with staying coherent for long periods of time. |
| Approach: | They propose a controlled generation task which expands a sequence of facts into a longer narrative. |
| Outcome: | The proposed model produces competitive fluency while adhering to the requested facts. |
Plan-then-Generate: Controlled Data-to-Text Generation via Planning (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies focus on producing results that are close to the references, i.e. what to generate and in what order (the output structure) cannot be explicitly controlled by the users. |
| Approach: | They propose a Plan-then-Generate framework to improve the controllability of neural data-to-text models. |
| Outcome: | The proposed model can control both the intra-sentence and inter-sentent structure of the generated output. |
Few-shot Table-to-text Generation with Prefix-Controlled Generator (2022.coling-1)
Copied to clipboard
| Challenge: | Neural table-to-text generation approaches are data-hungry and lack labeled data. |
| Approach: | They propose a prompt-based approach for few-shot table-to-text generation using a task-specific prefix and an input-specific input prefix. |
| Outcome: | The proposed approach is able to generate table-to-text summaries with a few instances and is validated on human, book and song datasets. |
Long Text Generation with Topic-aware Discrete Latent Variable Model (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent work focuses on the modeling of discourse relation, resulting in discrete codes learning shallow semantics. |
| Approach: | They propose a topic-aware latent code-guided text generation model that encourages discrete codes to model information about topics. |
| Outcome: | The proposed model generates more topic-relevant and coherent texts. |
Latent-Variable Generative Models for Data-Efficient Text Classification (D19-1)
Copied to clipboard
| Challenge: | Generative classifiers offer potential advantages over discriminative classifications, including data efficiency and zero-shot learning. |
| Approach: | They introduce discrete latent variables into generative story to improve classifiers' performance . they empirically characterize performance of their models on six text classification datasets . |
| Outcome: | The proposed model outperforms discriminative and generative classifiers on six text classification datasets. |